[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:14:09 dist-test-slave-ncp9 krb5kdc[17484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:14:09 dist-test-slave-ncp9 krb5kdc[17484](info): set up 2 sockets
Aug 11 09:14:09 dist-test-slave-ncp9 krb5kdc[17484](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 09:14:10.571646 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:14:10 dist-test-slave-ncp9 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:14:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:14:10.762827 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:45343
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:45343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:11.102407 17501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:11.102697 17501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:11.102819 17501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:11.110030 17501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:11.110124 17501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:11.110198 17501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:11.110268 17501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:11.110328 17501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:11.120374 17501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:45343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:45343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:11.122164 17501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:11.124135 17501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:11.141721 17513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:11.141625 17509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:11.141805 17508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:11.142464 17501 server_base.cc:1034] running on GCE node
I20260811 09:14:11.143535 17501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:11.145262 17501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:11.146447 17501 hybrid_clock.cc:648] HybridClock initialized: now 1786439651146343 us; error 129 us; skew 500 ppm
I20260811 09:14:11.146952 17501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:11 dist-test-slave-ncp9 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439651, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:11.150520 17501 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:14:11.153062 17501 webserver.cc:460] Webserver started at http://127.17.16.126:41915/ using document root <none> and password file <none>
I20260811 09:14:11.153914 17501 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:11.154049 17501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:11.154552 17501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:11.158557 17501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "30c14210582c4c1eb1e0396efe9f71dc"
format_stamp: "Formatted at 2026-08-11 09:14:11 on dist-test-slave-ncp9"
server_key: "19efbfe7e690661c0812520fda2c8707"
server_key_iv: "ad6e415f3ed62d21ba561aaaa573a7ed"
server_key_version: "encryptionkey@0"
I20260811 09:14:11.159366 17501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "30c14210582c4c1eb1e0396efe9f71dc"
format_stamp: "Formatted at 2026-08-11 09:14:11 on dist-test-slave-ncp9"
server_key: "19efbfe7e690661c0812520fda2c8707"
server_key_iv: "ad6e415f3ed62d21ba561aaaa573a7ed"
server_key_version: "encryptionkey@0"
I20260811 09:14:11.167362 17501 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.002s
I20260811 09:14:11.172855 17523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:11.174750 17501 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 09:14:11.175034 17501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "30c14210582c4c1eb1e0396efe9f71dc"
format_stamp: "Formatted at 2026-08-11 09:14:11 on dist-test-slave-ncp9"
server_key: "19efbfe7e690661c0812520fda2c8707"
server_key_iv: "ad6e415f3ed62d21ba561aaaa573a7ed"
server_key_version: "encryptionkey@0"
I20260811 09:14:11.175269 17501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:11.195859 17501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:11.200749 17501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:11.201097 17501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:11.225454 17501 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:45343
I20260811 09:14:11.225489 17602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:45343 every 8 connection(s)
I20260811 09:14:11.227552 17501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:11.229497 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 17501
I20260811 09:14:11.229773 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:11.231918 17473 external_mini_cluster.cc:1428] Setting key 33c595cdccba4c3622387825f006ad2d
I20260811 09:14:11.234316 17603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:11.255576 17603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc: Bootstrap starting.
I20260811 09:14:11.263063 17603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:11.265213 17603 log.cc:824] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:11.269340 17603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc: No bootstrap required, opened a new log
Aug 11 09:14:11 dist-test-slave-ncp9 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:11.281569 17606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:11.239494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.17.16.126:45343)
0811 09:14:11.241233 (+ 1739us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:11.241246 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:11.241290 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:14:11.253801 (+ 12511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:11.253823 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:11.254352 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:11.254759 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:11.259500 (+ 4741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:11.260999 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:11.263785 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:11.264542 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:11.264724 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:11.271265 (+ 6541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:11.271336 (+ 71us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:11.271357 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:11.271406 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:11.275507 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:11.276339 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:11.276348 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:11.276355 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:11.276731 (+ 376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:11.277240 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:11.277248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:11.277252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:11.278201 (+ 949us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:11.278774 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:11.279524 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 09:14:11.279973 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1456,"thread_start_us":1203,"threads_started":1}
I20260811 09:14:11.281803 17603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c14210582c4c1eb1e0396efe9f71dc" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45343 } }
I20260811 09:14:11.282390 17603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:11.282588 17603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c14210582c4c1eb1e0396efe9f71dc, State: Initialized, Role: FOLLOWER
I20260811 09:14:11.283804 17603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c14210582c4c1eb1e0396efe9f71dc" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45343 } }
I20260811 09:14:11.284164 17603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:11.284421 17603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:11.284659 17603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:11.287173 17603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c14210582c4c1eb1e0396efe9f71dc" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45343 } }
I20260811 09:14:11.288151 17603 leader_election.cc:302] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30c14210582c4c1eb1e0396efe9f71dc; no voters:
I20260811 09:14:11.289001 17603 leader_election.cc:288] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:11.289232 17608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:11.289947 17608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [term 1 LEADER]: Becoming Leader. State: Replica: 30c14210582c4c1eb1e0396efe9f71dc, State: Running, Role: LEADER
I20260811 09:14:11.290731 17608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c14210582c4c1eb1e0396efe9f71dc" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45343 } }
I20260811 09:14:11.291886 17603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:11.296597 17611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30c14210582c4c1eb1e0396efe9f71dc. Latest consensus state: current_term: 1 leader_uuid: "30c14210582c4c1eb1e0396efe9f71dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c14210582c4c1eb1e0396efe9f71dc" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45343 } } }
I20260811 09:14:11.297480 17611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:11.297067 17609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30c14210582c4c1eb1e0396efe9f71dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30c14210582c4c1eb1e0396efe9f71dc" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45343 } } }
I20260811 09:14:11.298060 17609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:11.299763 17628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:11.306864 17628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:11.321362 17628 catalog_manager.cc:1347] Generated new cluster ID: 2c392238ff1b46cabb817423d9951b6c
I20260811 09:14:11.321547 17628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:11.334936 17628 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:11.337015 17628 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:11.344877 17628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30c14210582c4c1eb1e0396efe9f71dc: Generated new TSK 0
I20260811 09:14:11.345839 17628 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:14:11.412393 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45343
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:11.759524 17652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:11.759876 17652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:11.760021 17652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:11.767218 17652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:11.767336 17652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:11.767554 17652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:14:11.777244 17652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45343
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:11.778503 17652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:11.780102 17652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:11.794672 17658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:11.794572 17659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:11.794601 17662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:11.795390 17652 server_base.cc:1034] running on GCE node
I20260811 09:14:11.795971 17652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:11.796970 17652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:11.798200 17652 hybrid_clock.cc:648] HybridClock initialized: now 1786439651798150 us; error 38 us; skew 500 ppm
I20260811 09:14:11.798592 17652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:11 dist-test-slave-ncp9 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439651, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:11.801795 17652 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:14:11.803158 17652 webserver.cc:460] Webserver started at http://127.17.16.65:35997/ using document root <none> and password file <none>
I20260811 09:14:11.803821 17652 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:11.803952 17652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:11.804311 17652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:11.807231 17652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "5080fcca303d419c87e05dc3e65f7cb2"
format_stamp: "Formatted at 2026-08-11 09:14:11 on dist-test-slave-ncp9"
server_key: "c38e755a140d4ffdef8777935539c270"
server_key_iv: "4b4351f4971db8d83e7d4f28fc0ae13c"
server_key_version: "encryptionkey@0"
I20260811 09:14:11.808066 17652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "5080fcca303d419c87e05dc3e65f7cb2"
format_stamp: "Formatted at 2026-08-11 09:14:11 on dist-test-slave-ncp9"
server_key: "c38e755a140d4ffdef8777935539c270"
server_key_iv: "4b4351f4971db8d83e7d4f28fc0ae13c"
server_key_version: "encryptionkey@0"
I20260811 09:14:11.813429 17652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:14:11.816854 17672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:11.818179 17652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:14:11.818315 17652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "5080fcca303d419c87e05dc3e65f7cb2"
format_stamp: "Formatted at 2026-08-11 09:14:11 on dist-test-slave-ncp9"
server_key: "c38e755a140d4ffdef8777935539c270"
server_key_iv: "4b4351f4971db8d83e7d4f28fc0ae13c"
server_key_version: "encryptionkey@0"
I20260811 09:14:11.818487 17652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:11.831444 17652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:11.835204 17652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:11.835477 17652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:11.836517 17652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:11.838179 17652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:11.838253 17652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:11.838407 17652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:11.838470 17652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:11.873677 17652 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:39291
I20260811 09:14:11.873719 17854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:39291 every 8 connection(s)
I20260811 09:14:11.875010 17652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:14:11.882117 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 17652
I20260811 09:14:11.882472 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:14:11.883153 17473 external_mini_cluster.cc:1428] Setting key e9a45f703e2765d7c5ad5db97f13e85a
Aug 11 09:14:11 dist-test-slave-ncp9 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439651, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:11.898452 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:11.878248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:55665 (local address 127.17.16.126:45343)
0811 09:14:11.878698 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:11.878702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:11.879839 (+ 1137us) server_negotiation.cc:407] Connection header received
0811 09:14:11.881815 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:11.881823 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:11.881916 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:11.882057 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:11.883549 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:11.884464 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:11.886296 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:11.886698 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:11.886820 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:11.890729 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:11.890737 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:11.890742 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:11.890785 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:11.894195 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:11.895151 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:11.895158 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:11.895161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:11.895315 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:11.895891 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:11.895896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:11.895899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:11.896283 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:11.896427 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:11.897742 (+ 1315us) server_negotiation.cc:299] Negotiation successful
0811 09:14:11.898002 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":219,"threads_started":1}
I20260811 09:14:11.899380 17864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:11.878846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:45343 (local address 127.17.16.65:55665)
0811 09:14:11.879681 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:11.879726 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:11.881533 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:11.882231 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:11.882239 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:11.882772 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:11.883367 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:11.883387 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:11.884641 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:11.884645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:11.886154 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:11.886168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:11.886931 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:11.886940 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:11.887175 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:11.887877 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:11.887899 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:11.890421 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:11.894361 (+ 3940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:11.894368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:11.894382 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:11.895000 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:11.895460 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:11.895465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:11.895469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:11.895746 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:11.896466 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:11.896477 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:11.897113 (+ 636us) client_negotiation.cc:769] Sending connection context
0811 09:14:11.897504 (+ 391us) client_negotiation.cc:240] Negotiation successful
0811 09:14:11.898073 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":355,"threads_started":1}
I20260811 09:14:11.901289 17855 heartbeater.cc:347] Connected to a master server at 127.17.16.126:45343
I20260811 09:14:11.901795 17855 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:11.902690 17855 heartbeater.cc:511] Master 127.17.16.126:45343 requested a full tablet report, sending...
I20260811 09:14:11.906709 17556 ts_manager.cc:194] Registered new tserver with Master: 5080fcca303d419c87e05dc3e65f7cb2 (127.17.16.65:39291)
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
I20260811 09:14:11.908875 17556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:55665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:14:11.936568 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45343
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:12.273650 17867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:12.273938 17867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:12.274061 17867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:12.281562 17867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:12.281682 17867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:12.281816 17867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:14:12.291800 17867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45343
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:12.293216 17867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:12.294696 17867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:12.308476 17878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:12.308629 17882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:12.308741 17879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:12.309746 17867 server_base.cc:1034] running on GCE node
I20260811 09:14:12.310355 17867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:12.311245 17867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:12.312444 17867 hybrid_clock.cc:648] HybridClock initialized: now 1786439652312372 us; error 59 us; skew 500 ppm
I20260811 09:14:12.312717 17867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:12 dist-test-slave-ncp9 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439652, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:12.315867 17867 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:14:12.317448 17867 webserver.cc:460] Webserver started at http://127.17.16.66:44001/ using document root <none> and password file <none>
I20260811 09:14:12.318089 17867 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:12.318186 17867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:12.318441 17867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:12.321324 17867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "8c89fdf100fc402496fbd2ae4b13ef81"
format_stamp: "Formatted at 2026-08-11 09:14:12 on dist-test-slave-ncp9"
server_key: "880f9d91a26538cef32b120046df1bc5"
server_key_iv: "175f2087788e0ceb705a8385ed06af61"
server_key_version: "encryptionkey@0"
I20260811 09:14:12.322242 17867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "8c89fdf100fc402496fbd2ae4b13ef81"
format_stamp: "Formatted at 2026-08-11 09:14:12 on dist-test-slave-ncp9"
server_key: "880f9d91a26538cef32b120046df1bc5"
server_key_iv: "175f2087788e0ceb705a8385ed06af61"
server_key_version: "encryptionkey@0"
I20260811 09:14:12.327775 17867 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:14:12.331147 17895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:12.332417 17867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:14:12.332564 17867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "8c89fdf100fc402496fbd2ae4b13ef81"
format_stamp: "Formatted at 2026-08-11 09:14:12 on dist-test-slave-ncp9"
server_key: "880f9d91a26538cef32b120046df1bc5"
server_key_iv: "175f2087788e0ceb705a8385ed06af61"
server_key_version: "encryptionkey@0"
I20260811 09:14:12.332742 17867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:12.356309 17867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:12.360136 17867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:12.360409 17867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:12.361388 17867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:12.362684 17867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:12.362764 17867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:12.362890 17867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:12.362946 17867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:12.398862 17867 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:40963
I20260811 09:14:12.398896 18084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:40963 every 8 connection(s)
I20260811 09:14:12.400195 17867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:14:12.404937 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 17867
I20260811 09:14:12.405113 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:14:12.405517 17473 external_mini_cluster.cc:1428] Setting key a225b7bb884f12e4d901382a6cf531ef
Aug 11 09:14:12 dist-test-slave-ncp9 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439652, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:12.419631 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:12.402810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:36965 (local address 127.17.16.126:45343)
0811 09:14:12.403238 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:12.403243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:12.404157 (+ 914us) server_negotiation.cc:407] Connection header received
0811 09:14:12.406090 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:12.406097 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:12.406176 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:12.406272 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:12.407719 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:12.408429 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:12.409984 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:12.410422 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:12.410533 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:12.413924 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:12.413929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:12.413932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:12.413964 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:12.416426 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:12.417243 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:12.417248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:12.417252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:12.417411 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:12.417862 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:12.417867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:12.417870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:12.418226 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:12.418368 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:12.419054 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 09:14:12.419278 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":224,"threads_started":1}
I20260811 09:14:12.420433 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:12.403228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:45343 (local address 127.17.16.66:36965)
0811 09:14:12.404013 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:12.404049 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:12.405838 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:12.406513 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:12.406521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:12.407007 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:12.407550 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:12.407567 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:12.408573 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:12.408577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:12.409845 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:12.409858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:12.410635 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:12.410643 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:12.410873 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:12.411529 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:12.411551 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:12.413708 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:12.416561 (+ 2853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:12.416571 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:12.416587 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:12.417110 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:12.417525 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:12.417529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:12.417532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:12.417745 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:12.418345 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:12.418351 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:12.418672 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 09:14:12.418901 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 09:14:12.419316 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":371,"threads_started":1}
I20260811 09:14:12.421859 18085 heartbeater.cc:347] Connected to a master server at 127.17.16.126:45343
I20260811 09:14:12.422204 18085 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:12.422925 18085 heartbeater.cc:511] Master 127.17.16.126:45343 requested a full tablet report, sending...
I20260811 09:14:12.424508 17556 ts_manager.cc:194] Registered new tserver with Master: 8c89fdf100fc402496fbd2ae4b13ef81 (127.17.16.66:40963)
I20260811 09:14:12.425375 17556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:36965
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:14:12.451946 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45343
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:12.783927 18093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:12.784251 18093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:12.784494 18093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:12.792392 18093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:12.792511 18093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:12.792671 18093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:14:12.802413 18093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45343
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:12.803711 18093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:12.805115 18093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:12.818470 18098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:12.818491 18107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:12.818450 18099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:12.819415 18093 server_base.cc:1034] running on GCE node
I20260811 09:14:12.820088 18093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:12.821257 18093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:12.822481 18093 hybrid_clock.cc:648] HybridClock initialized: now 1786439652822417 us; error 54 us; skew 500 ppm
I20260811 09:14:12.822805 18093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:12 dist-test-slave-ncp9 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439652, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:12.826153 18093 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:14:12.827770 18093 webserver.cc:460] Webserver started at http://127.17.16.67:44055/ using document root <none> and password file <none>
I20260811 09:14:12.828424 18093 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:12.828526 18093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:12.828801 18093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:12.832038 18093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "b15ca01c07d6417fb4f491cc35dfe983"
format_stamp: "Formatted at 2026-08-11 09:14:12 on dist-test-slave-ncp9"
server_key: "a75029aa2ef7fdb58302a81131223073"
server_key_iv: "c19458b1c389d7836f89a24b340a590b"
server_key_version: "encryptionkey@0"
I20260811 09:14:12.832849 18093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "b15ca01c07d6417fb4f491cc35dfe983"
format_stamp: "Formatted at 2026-08-11 09:14:12 on dist-test-slave-ncp9"
server_key: "a75029aa2ef7fdb58302a81131223073"
server_key_iv: "c19458b1c389d7836f89a24b340a590b"
server_key_version: "encryptionkey@0"
I20260811 09:14:12.838332 18093 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 09:14:12.841749 18118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:12.843005 18093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:14:12.843142 18093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "b15ca01c07d6417fb4f491cc35dfe983"
format_stamp: "Formatted at 2026-08-11 09:14:12 on dist-test-slave-ncp9"
server_key: "a75029aa2ef7fdb58302a81131223073"
server_key_iv: "c19458b1c389d7836f89a24b340a590b"
server_key_version: "encryptionkey@0"
I20260811 09:14:12.843294 18093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:12.861902 18093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:12.866080 18093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:12.866328 18093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:12.867537 18093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:12.868911 18093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:12.868986 18093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:12.869096 18093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:12.869140 18093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:12.906986 18093 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:44727
I20260811 09:14:12.907032 18303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:44727 every 8 connection(s)
I20260811 09:14:12.908245 18093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:14:12.909411 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 18093
I20260811 09:14:12.909607 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:14:12.910023 17473 external_mini_cluster.cc:1428] Setting key 8d7a038004ddd79fa928823b1b081a59
I20260811 09:14:12.912580 17855 heartbeater.cc:503] Master 127.17.16.126:45343 was elected leader, sending a full tablet report...
Aug 11 09:14:12 dist-test-slave-ncp9 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439652, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:12.928622 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:12.910968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:43609 (local address 127.17.16.126:45343)
0811 09:14:12.911175 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:12.911181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:12.912830 (+ 1649us) server_negotiation.cc:407] Connection header received
0811 09:14:12.914906 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:12.914912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:12.914992 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:12.915080 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:12.916535 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:12.917409 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:12.919007 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:12.919421 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:12.919527 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:12.922987 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:12.922992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:12.922995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:12.923027 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:12.925589 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:12.926373 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:12.926378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:12.926381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:12.926513 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:12.926940 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:12.926946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:12.926948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:12.927243 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:12.927387 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:12.928055 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 09:14:12.928285 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:14:12.929574 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:12.911525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:45343 (local address 127.17.16.67:43609)
0811 09:14:12.912544 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:12.912604 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:12.914626 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:12.915325 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:12.915332 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:12.915823 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:12.916375 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:12.916393 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:12.917567 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:12.917571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:12.918870 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:12.918883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:12.919637 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:12.919645 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:12.919859 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:12.920437 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:12.920458 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:12.922797 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:12.925722 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:12.925731 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:12.925747 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:12.926251 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:12.926620 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:12.926626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:12.926630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:12.926830 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:12.927371 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:12.927378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:12.927696 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 09:14:12.927943 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 09:14:12.928456 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":469,"threads_started":1}
I20260811 09:14:12.930853 18304 heartbeater.cc:347] Connected to a master server at 127.17.16.126:45343
I20260811 09:14:12.931137 18304 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:12.931869 18304 heartbeater.cc:511] Master 127.17.16.126:45343 requested a full tablet report, sending...
I20260811 09:14:12.933569 17555 ts_manager.cc:194] Registered new tserver with Master: b15ca01c07d6417fb4f491cc35dfe983 (127.17.16.67:44727)
I20260811 09:14:12.934477 17555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:43609
I20260811 09:14:12.939704 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:14:12 dist-test-slave-ncp9 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439652, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:14:12 dist-test-slave-ncp9 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439652, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:14:12.991557 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:12.975692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.17.16.65:39291)
0811 09:14:12.976348 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:12.976356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:12.976404 (+ 48us) server_negotiation.cc:407] Connection header received
0811 09:14:12.976466 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:12.976472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:12.976722 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:12.976844 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:12.978274 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:12.979670 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:12.981176 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:12.981797 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:12.981895 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:12.986089 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:12.986101 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:12.986117 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:12.986157 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:12.989040 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:12.989614 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:12.989621 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:12.989624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:12.989750 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:12.990032 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:12.990035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:12.990037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:12.990419 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:12.990659 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:12.990966 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 09:14:12.991247 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":354,"threads_started":1}
W20260811 09:14:12.993052 17718 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44800
Aug 11 09:14:13 dist-test-slave-ncp9 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:14:13 dist-test-slave-ncp9 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:14:13.040129 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:13.027379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.17.16.65:39291)
0811 09:14:13.027606 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:13.027613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:13.027865 (+ 252us) server_negotiation.cc:407] Connection header received
0811 09:14:13.027996 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:13.028000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:13.028076 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:13.028156 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:13.029442 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:13.030172 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:13.031379 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:13.031848 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:13.031947 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:13.035501 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:13.035506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:13.035509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:13.035547 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:13.038088 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:13.038652 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:13.038656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:13.038658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:13.038780 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:13.039052 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:13.039055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:13.039057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:13.039287 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:13.039461 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:13.039666 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 09:14:13.039829 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:14:13.041878 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 17652
I20260811 09:14:13.054253 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:13.042985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.17.16.65:39291)
0811 09:14:13.043174 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:13.043179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:13.043337 (+ 158us) server_negotiation.cc:407] Connection header received
0811 09:14:13.043476 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:13.043479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:13.043535 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:13.043613 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:13.044720 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:13.045429 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:13.046640 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:13.047081 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:13.047185 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:13.049244 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:13.049249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:13.049252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:13.049279 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:13.051812 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:13.052346 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:13.052349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:13.052351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:13.052476 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:13.052886 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:13.052933 (+ 47us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:13.052936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:13.053237 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:13.053429 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:13.053750 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 09:14:13.053923 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:14:13.055073 17849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:13.136878 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 17652
I20260811 09:14:13.159469 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 17867
Aug 11 09:14:13 dist-test-slave-ncp9 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:14:13.174741 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:13.160338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51538 (local address 127.17.16.66:40963)
0811 09:14:13.160865 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:13.160872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:13.160918 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:14:13.160990 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:13.160997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:13.161206 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:13.161325 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:13.162789 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:13.163819 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:13.165112 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:13.165663 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:13.165758 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:13.169467 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:13.169478 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:13.169492 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:13.169534 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:13.172276 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:13.172880 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:13.172886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:13.172911 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:13.173040 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:13.173362 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:13.173365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:13.173367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:13.173712 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:13.173957 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:13.174170 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 09:14:13.174374 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":252,"threads_started":1}
I20260811 09:14:13.176007 18077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:13.260493 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 17867
I20260811 09:14:13.285203 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 18093
Aug 11 09:14:13 dist-test-slave-ncp9 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:14:13.301944 18333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:13.286385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49704 (local address 127.17.16.67:44727)
0811 09:14:13.287093 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:13.287100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:13.287129 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:14:13.287205 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:13.287212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:13.287412 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:13.287538 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:13.288821 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:13.290316 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:13.291594 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:13.292205 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:13.292316 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:13.296423 (+ 4107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:13.296434 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:13.296449 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:13.296483 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:13.299190 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:13.299822 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:13.299832 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:13.299835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:13.299981 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:13.300341 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:13.300347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:13.300350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:13.300721 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:13.301034 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:13.301345 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 09:14:13.301608 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":382,"threads_started":1}
I20260811 09:14:13.303272 18297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:13.385960 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 18093
I20260811 09:14:13.404964 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 17501
I20260811 09:14:13.406181 17595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:13.511543 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 17501
2026-08-11T09:14:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4233 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:14:13 dist-test-slave-ncp9 krb5kdc[18346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:14:13 dist-test-slave-ncp9 krb5kdc[18346](info): set up 2 sockets
Aug 11 09:14:13 dist-test-slave-ncp9 krb5kdc[18346](info): commencing operation
krb5kdc: starting...
W20260811 09:14:14.713985 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:14:14 dist-test-slave-ncp9 krb5kdc[18346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:14:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:14:14.898371 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46273
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:46273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:15.215667 18364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:15.215950 18364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:15.216081 18364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:15.223245 18364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:15.223362 18364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:15.223445 18364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:15.223526 18364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:15.223621 18364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:15.233261 18364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:46273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:15.234485 18364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:15.235752 18364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:15.247033 18374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:15.247123 18376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:15.247099 18380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:15.247798 18364 server_base.cc:1034] running on GCE node
I20260811 09:14:15.248540 18364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:15.249838 18364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:15.251034 18364 hybrid_clock.cc:648] HybridClock initialized: now 1786439655250995 us; error 25 us; skew 500 ppm
I20260811 09:14:15.251333 18364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:15 dist-test-slave-ncp9 krb5kdc[18346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439655, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:15.254187 18364 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:14:15.255600 18364 webserver.cc:460] Webserver started at http://127.17.16.126:43951/ using document root <none> and password file <none>
I20260811 09:14:15.256227 18364 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:15.256345 18364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:15.256645 18364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:15.259595 18364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "70626ed15b6b434cad57513fa59cdf4e"
format_stamp: "Formatted at 2026-08-11 09:14:15 on dist-test-slave-ncp9"
server_key: "617b27f933b7a69c7dc2d72631f12939"
server_key_iv: "348ad88905ea8ab76e3f0b688408e2b5"
server_key_version: "encryptionkey@0"
I20260811 09:14:15.260306 18364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "70626ed15b6b434cad57513fa59cdf4e"
format_stamp: "Formatted at 2026-08-11 09:14:15 on dist-test-slave-ncp9"
server_key: "617b27f933b7a69c7dc2d72631f12939"
server_key_iv: "348ad88905ea8ab76e3f0b688408e2b5"
server_key_version: "encryptionkey@0"
I20260811 09:14:15.265383 18364 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 09:14:15.268702 18392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:15.269946 18364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:14:15.270109 18364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "70626ed15b6b434cad57513fa59cdf4e"
format_stamp: "Formatted at 2026-08-11 09:14:15 on dist-test-slave-ncp9"
server_key: "617b27f933b7a69c7dc2d72631f12939"
server_key_iv: "348ad88905ea8ab76e3f0b688408e2b5"
server_key_version: "encryptionkey@0"
I20260811 09:14:15.270285 18364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:15.290735 18364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:15.294517 18364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:15.294788 18364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:15.312059 18364 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:46273
I20260811 09:14:15.312083 18466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:46273 every 8 connection(s)
I20260811 09:14:15.313369 18364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:15.314088 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 18364
I20260811 09:14:15.314328 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:15.315028 17473 external_mini_cluster.cc:1428] Setting key 4b510dd3199d8cb657e8fd0c1bdb0313
I20260811 09:14:15.317812 18470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:14:15 dist-test-slave-ncp9 krb5kdc[18346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:15.330257 18470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e: Bootstrap starting.
I20260811 09:14:15.333384 18470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:15.334599 18470 log.cc:824] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:15.337936 18470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e: No bootstrap required, opened a new log
I20260811 09:14:15.337970 18474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:15.316881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.17.16.126:46273)
0811 09:14:15.317714 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:15.317728 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:15.317781 (+ 53us) server_negotiation.cc:407] Connection header received
0811 09:14:15.320031 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:15.320058 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:15.320511 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:15.321002 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:15.322331 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:15.324396 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:15.325608 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:15.326609 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:15.326783 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:15.330675 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:15.330683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:15.330696 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:15.330732 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:15.333656 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:15.334425 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:15.334431 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:15.334437 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:15.334591 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:15.334998 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:15.335003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:15.335005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:15.335809 (+ 804us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:15.336151 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:15.336558 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 09:14:15.336973 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":326,"threads_started":1}
I20260811 09:14:15.346306 18470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70626ed15b6b434cad57513fa59cdf4e" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46273 } }
I20260811 09:14:15.346705 18470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:15.346823 18470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70626ed15b6b434cad57513fa59cdf4e, State: Initialized, Role: FOLLOWER
I20260811 09:14:15.347422 18470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70626ed15b6b434cad57513fa59cdf4e" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46273 } }
I20260811 09:14:15.347641 18470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:15.347774 18470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:15.347937 18470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:15.350262 18470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70626ed15b6b434cad57513fa59cdf4e" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46273 } }
I20260811 09:14:15.350726 18470 leader_election.cc:302] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70626ed15b6b434cad57513fa59cdf4e; no voters:
I20260811 09:14:15.351282 18470 leader_election.cc:288] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:15.351400 18476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:15.352013 18476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [term 1 LEADER]: Becoming Leader. State: Replica: 70626ed15b6b434cad57513fa59cdf4e, State: Running, Role: LEADER
I20260811 09:14:15.352540 18476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70626ed15b6b434cad57513fa59cdf4e" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46273 } }
I20260811 09:14:15.353278 18470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:15.355332 18477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70626ed15b6b434cad57513fa59cdf4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70626ed15b6b434cad57513fa59cdf4e" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46273 } } }
I20260811 09:14:15.355697 18477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:15.356273 18478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70626ed15b6b434cad57513fa59cdf4e. Latest consensus state: current_term: 1 leader_uuid: "70626ed15b6b434cad57513fa59cdf4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70626ed15b6b434cad57513fa59cdf4e" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46273 } } }
I20260811 09:14:15.356673 18478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:15.365885 18487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:15.369900 18487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:15.378055 18487 catalog_manager.cc:1347] Generated new cluster ID: 90b686d0a94e47e2b1df9e1fb7e60b9e
I20260811 09:14:15.378171 18487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:15.394202 18487 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:15.395442 18487 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:15.402441 18487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70626ed15b6b434cad57513fa59cdf4e: Generated new TSK 0
I20260811 09:14:15.403188 18487 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:14:15.456308 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46273
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:15.800654 18513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:15.800951 18513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:15.801057 18513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:15.808074 18513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:15.808171 18513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:15.808295 18513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:14:15.817680 18513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46273
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:15.818893 18513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:15.820231 18513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:15.833868 18524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:15.833910 18520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:15.833940 18521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:15.834710 18513 server_base.cc:1034] running on GCE node
I20260811 09:14:15.835314 18513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:15.836249 18513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:15.837441 18513 hybrid_clock.cc:648] HybridClock initialized: now 1786439655837398 us; error 38 us; skew 500 ppm
I20260811 09:14:15.837751 18513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:15 dist-test-slave-ncp9 krb5kdc[18346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439655, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:15.840842 18513 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:14:15.842291 18513 webserver.cc:460] Webserver started at http://127.17.16.65:42241/ using document root <none> and password file <none>
I20260811 09:14:15.842957 18513 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:15.843058 18513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:15.843328 18513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:15.846330 18513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "407e3e411eed4013a87f3b2286118090"
format_stamp: "Formatted at 2026-08-11 09:14:15 on dist-test-slave-ncp9"
server_key: "e13544514ee70f970e352160166ccff4"
server_key_iv: "75e151df7edc6bdbd49172f103531511"
server_key_version: "encryptionkey@0"
I20260811 09:14:15.847062 18513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "407e3e411eed4013a87f3b2286118090"
format_stamp: "Formatted at 2026-08-11 09:14:15 on dist-test-slave-ncp9"
server_key: "e13544514ee70f970e352160166ccff4"
server_key_iv: "75e151df7edc6bdbd49172f103531511"
server_key_version: "encryptionkey@0"
I20260811 09:14:15.851997 18513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:14:15.855566 18544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:15.856938 18513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:14:15.857087 18513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "407e3e411eed4013a87f3b2286118090"
format_stamp: "Formatted at 2026-08-11 09:14:15 on dist-test-slave-ncp9"
server_key: "e13544514ee70f970e352160166ccff4"
server_key_iv: "75e151df7edc6bdbd49172f103531511"
server_key_version: "encryptionkey@0"
I20260811 09:14:15.857290 18513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:15.879321 18513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:15.883721 18513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:15.883991 18513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:15.884990 18513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:15.886351 18513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:15.886428 18513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:15.886543 18513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:15.886593 18513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:15.917587 18513 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:33047
I20260811 09:14:15.917665 18721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:33047 every 8 connection(s)
I20260811 09:14:15.918828 18513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:14:15.927096 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 18513
I20260811 09:14:15.927294 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:14:15.927947 17473 external_mini_cluster.cc:1428] Setting key cb1f6e7b64cd25bd241f0b4a3c46e5de
Aug 11 09:14:15 dist-test-slave-ncp9 krb5kdc[18346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439655, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:15.938513 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:15.921222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:46451 (local address 127.17.16.126:46273)
0811 09:14:15.921659 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:15.921665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:15.922615 (+ 950us) server_negotiation.cc:407] Connection header received
0811 09:14:15.924485 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:15.924491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:15.924561 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:15.924647 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:15.926093 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:15.926792 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:15.928379 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:15.928887 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:15.929046 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:15.932983 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:15.932988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:15.932992 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:15.933038 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:15.935389 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:15.936196 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:15.936200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:15.936202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:15.936317 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:15.936776 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:15.936781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:15.936784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:15.937158 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:15.937305 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:15.937964 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 09:14:15.938192 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":193,"threads_started":1}
I20260811 09:14:15.939328 18733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:15.921751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46273 (local address 127.17.16.65:46451)
0811 09:14:15.922455 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:15.922502 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:15.924219 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:15.924830 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:15.924838 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:15.925383 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:15.925919 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:15.925935 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:15.926946 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:15.926950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:15.928242 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:15.928255 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:15.929107 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:15.929112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:15.929324 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:15.930039 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:15.930069 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:15.932725 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:15.935526 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:15.935535 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:15.935552 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:15.936085 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:15.936408 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:15.936412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:15.936415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:15.936649 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:15.937269 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:15.937275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:15.937591 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 09:14:15.937824 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 09:14:15.938225 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":257,"threads_started":1}
I20260811 09:14:15.940847 18727 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46273
I20260811 09:14:15.941197 18727 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:15.942055 18727 heartbeater.cc:511] Master 127.17.16.126:46273 requested a full tablet report, sending...
I20260811 09:14:15.944614 18419 ts_manager.cc:194] Registered new tserver with Master: 407e3e411eed4013a87f3b2286118090 (127.17.16.65:33047)
I20260811 09:14:15.946699 18419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:46451
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:14:15.973587 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46273
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:16.306365 18736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:16.306644 18736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:16.306784 18736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:16.313902 18736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:16.314011 18736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:16.314163 18736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:14:16.323736 18736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46273
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:16.325078 18736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:16.326388 18736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:16.338836 18747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:16.338801 18745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:16.339012 18744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:16.339653 18736 server_base.cc:1034] running on GCE node
I20260811 09:14:16.340168 18736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:16.341037 18736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:16.342232 18736 hybrid_clock.cc:648] HybridClock initialized: now 1786439656342179 us; error 44 us; skew 500 ppm
I20260811 09:14:16.342497 18736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:16 dist-test-slave-ncp9 krb5kdc[18346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:16.345496 18736 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:14:16.346830 18736 webserver.cc:460] Webserver started at http://127.17.16.66:43977/ using document root <none> and password file <none>
I20260811 09:14:16.347477 18736 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:16.347591 18736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:16.347883 18736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:16.350801 18736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "34e3e292e88c496eaeb5031f4bd042ab"
format_stamp: "Formatted at 2026-08-11 09:14:16 on dist-test-slave-ncp9"
server_key: "fd0d3c8d66e63b228251e2b3cb474dd3"
server_key_iv: "807af12d2f48baa75dbc81d31d2e9c23"
server_key_version: "encryptionkey@0"
I20260811 09:14:16.351599 18736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "34e3e292e88c496eaeb5031f4bd042ab"
format_stamp: "Formatted at 2026-08-11 09:14:16 on dist-test-slave-ncp9"
server_key: "fd0d3c8d66e63b228251e2b3cb474dd3"
server_key_iv: "807af12d2f48baa75dbc81d31d2e9c23"
server_key_version: "encryptionkey@0"
I20260811 09:14:16.356580 18736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:14:16.360119 18759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:16.361485 18736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:14:16.361624 18736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "34e3e292e88c496eaeb5031f4bd042ab"
format_stamp: "Formatted at 2026-08-11 09:14:16 on dist-test-slave-ncp9"
server_key: "fd0d3c8d66e63b228251e2b3cb474dd3"
server_key_iv: "807af12d2f48baa75dbc81d31d2e9c23"
server_key_version: "encryptionkey@0"
I20260811 09:14:16.361862 18736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:16.374401 18736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:16.378161 18736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:16.378399 18736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:16.379278 18736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:16.380550 18736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:16.380632 18736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:16.380720 18736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:16.380766 18736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:16.410533 18736 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:41323
I20260811 09:14:16.410576 18951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:41323 every 8 connection(s)
I20260811 09:14:16.411731 18736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:14:16.412741 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 18736
I20260811 09:14:16.412964 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:14:16.413535 17473 external_mini_cluster.cc:1428] Setting key d72716a74ccc1108a87bc899e16d67f9
Aug 11 09:14:16 dist-test-slave-ncp9 krb5kdc[18346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:16.434083 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.414411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:33153 (local address 127.17.16.126:46273)
0811 09:14:16.414640 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:16.414646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:16.415911 (+ 1265us) server_negotiation.cc:407] Connection header received
0811 09:14:16.418460 (+ 2549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:16.418465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:16.418533 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:16.418614 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:16.420586 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.421355 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.423352 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.424036 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.424217 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.427828 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:16.427834 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:16.427838 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:16.427876 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:16.431042 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:16.431722 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:16.431728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:16.431731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:16.431901 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:16.432284 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:16.432289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:16.432292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:16.432649 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:16.432805 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:16.433465 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 09:14:16.433717 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
I20260811 09:14:16.434823 18956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.414954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46273 (local address 127.17.16.66:33153)
0811 09:14:16.415735 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:16.415783 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:16.418154 (+ 2371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:16.418876 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:16.418887 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:16.419620 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:16.420413 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:16.420440 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.421508 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.421514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:16.423210 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:16.423223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.424372 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.424381 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:16.424631 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.425303 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:16.425325 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:16.427575 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:16.431182 (+ 3607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:16.431189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:16.431201 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:16.431600 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:16.431997 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:16.432000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:16.432002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:16.432181 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:16.432756 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:16.432761 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:16.433111 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 09:14:16.433339 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 09:14:16.433731 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":306,"threads_started":1}
I20260811 09:14:16.436206 18953 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46273
I20260811 09:14:16.436512 18953 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
I20260811 09:14:16.437325 18953 heartbeater.cc:511] Master 127.17.16.126:46273 requested a full tablet report, sending...
I20260811 09:14:16.438891 18419 ts_manager.cc:194] Registered new tserver with Master: 34e3e292e88c496eaeb5031f4bd042ab (127.17.16.66:41323)
I20260811 09:14:16.439707 18419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:33153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:14:16.458158 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46273
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:33405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:16.774248 18964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:16.774513 18964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:16.774644 18964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:16.781747 18964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:16.781852 18964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:16.782006 18964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:14:16.791522 18964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:33405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46273
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:16.792771 18964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:16.794090 18964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:16.806607 18969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:16.806586 18975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:16.806552 18971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:16.807330 18964 server_base.cc:1034] running on GCE node
I20260811 09:14:16.807888 18964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:16.808724 18964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:16.809963 18964 hybrid_clock.cc:648] HybridClock initialized: now 1786439656809887 us; error 47 us; skew 500 ppm
I20260811 09:14:16.810304 18964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:16 dist-test-slave-ncp9 krb5kdc[18346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:16.813472 18964 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:14:16.814966 18964 webserver.cc:460] Webserver started at http://127.17.16.67:35551/ using document root <none> and password file <none>
I20260811 09:14:16.815657 18964 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:16.815784 18964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:16.816118 18964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:16.819170 18964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "a7ffe18bd1594e478e73835e0aed507c"
format_stamp: "Formatted at 2026-08-11 09:14:16 on dist-test-slave-ncp9"
server_key: "0eb80336e721d324df2f66e08b4082ef"
server_key_iv: "81b43107f11577ab25af62e887493d7d"
server_key_version: "encryptionkey@0"
I20260811 09:14:16.820024 18964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "a7ffe18bd1594e478e73835e0aed507c"
format_stamp: "Formatted at 2026-08-11 09:14:16 on dist-test-slave-ncp9"
server_key: "0eb80336e721d324df2f66e08b4082ef"
server_key_iv: "81b43107f11577ab25af62e887493d7d"
server_key_version: "encryptionkey@0"
I20260811 09:14:16.825758 18964 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:14:16.829257 18990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:16.830639 18964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:14:16.830794 18964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "a7ffe18bd1594e478e73835e0aed507c"
format_stamp: "Formatted at 2026-08-11 09:14:16 on dist-test-slave-ncp9"
server_key: "0eb80336e721d324df2f66e08b4082ef"
server_key_iv: "81b43107f11577ab25af62e887493d7d"
server_key_version: "encryptionkey@0"
I20260811 09:14:16.830978 18964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:16.845561 18964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:16.849249 18964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:16.849491 18964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:16.850375 18964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:16.851611 18964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:16.851686 18964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:16.851800 18964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:16.851852 18964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:16.881515 18964 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:42183
I20260811 09:14:16.881560 19178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:42183 every 8 connection(s)
I20260811 09:14:16.882778 18964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:14:16.883683 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 18964
I20260811 09:14:16.883874 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:14:16.884415 17473 external_mini_cluster.cc:1428] Setting key 2492291ccd0bf90ef5054ccaa16aa8c5
Aug 11 09:14:16 dist-test-slave-ncp9 krb5kdc[18346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439656, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:16.902478 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.885587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:46679 (local address 127.17.16.126:46273)
0811 09:14:16.885780 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:16.885786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:16.887279 (+ 1493us) server_negotiation.cc:407] Connection header received
0811 09:14:16.889253 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:16.889258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:16.889349 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:16.889430 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:16.890972 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.891757 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.893239 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.893625 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.893716 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.896794 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:16.896799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:16.896803 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:16.896835 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:16.899631 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:16.900335 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:16.900338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:16.900340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:16.900474 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:16.900996 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:16.900999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:16.901001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:16.901279 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:16.901398 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:16.902037 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 09:14:16.902226 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:14:16.903380 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.886183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46273 (local address 127.17.16.67:46679)
0811 09:14:16.887102 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:16.887158 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:16.889025 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:16.889632 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:16.889639 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:16.890130 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:16.890844 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:16.890864 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.891881 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.891885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:16.893136 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:16.893148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.893789 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.893794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:16.893983 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.894539 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:16.894560 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:16.896612 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:16.899772 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:16.899779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:16.899791 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:16.900237 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:16.900599 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:16.900603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:16.900606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:16.900869 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:16.901381 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:16.901387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:16.901737 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 09:14:16.901954 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 09:14:16.902349 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":405,"threads_started":1}
I20260811 09:14:16.904613 19182 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46273
I20260811 09:14:16.904861 19182 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:16.905583 19182 heartbeater.cc:511] Master 127.17.16.126:46273 requested a full tablet report, sending...
I20260811 09:14:16.906903 18419 ts_manager.cc:194] Registered new tserver with Master: a7ffe18bd1594e478e73835e0aed507c (127.17.16.67:42183)
I20260811 09:14:16.907673 18419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:46679
I20260811 09:14:16.912215 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:14:16.932686 18732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.920142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51604 (local address 127.17.16.126:46273)
0811 09:14:16.920296 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:16.920301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:16.920998 (+ 697us) server_negotiation.cc:407] Connection header received
0811 09:14:16.921141 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:16.921146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:16.921225 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:16.921323 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:16.922728 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.923446 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.924710 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.925233 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.925325 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.927944 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:16.927948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:16.927951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:16.927979 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:16.930394 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:16.930952 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:16.930956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:16.930958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:16.931083 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:16.931446 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:16.931449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:16.931450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:16.931756 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:16.931930 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:16.932208 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 09:14:16.932376 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:14:16.941448 18419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:14:16.949676 18419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:16.950165 18727 heartbeater.cc:503] Master 127.17.16.126:46273 was elected leader, sending a full tablet report...
I20260811 09:14:16.991869 19210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.966832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46940 (local address 127.17.16.66:41323)
0811 09:14:16.982821 (+ 15989us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:16.982831 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:16.984306 (+ 1475us) server_negotiation.cc:407] Connection header received
0811 09:14:16.984412 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:16.984420 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:16.984655 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:16.984839 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:16.985784 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.987095 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.989840 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.990773 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.990896 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.991108 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:16.991390 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 09:14:16.991569 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15804,"thread_start_us":16009,"threads_started":1}
I20260811 09:14:16.991940 19205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.966275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:41323 (local address 127.0.0.1:46940)
0811 09:14:16.984049 (+ 17774us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:16.984066 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:16.984179 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:16.985141 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:16.985146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:16.985170 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:16.985585 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:16.985599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.987240 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.987245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:16.989683 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:16.989696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.990988 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.990995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:16.991168 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.991205 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 09:14:16.991307 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 09:14:16.991509 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17639,"mutex_wait_us":21,"thread_start_us":16893,"threads_started":1}
I20260811 09:14:16.992657 19208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.965876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:42183 (local address 127.0.0.1:46260)
0811 09:14:16.983428 (+ 17552us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:16.983443 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:16.983569 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:16.984570 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:16.984577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:16.984607 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:16.985176 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:16.985189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.987473 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.987477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:16.989601 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:16.989618 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.991844 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.991851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:16.992047 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.992079 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:14:16.992158 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 09:14:16.992282 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17426,"mutex_wait_us":1,"thread_start_us":17917,"threads_started":1}
I20260811 09:14:16.993441 19209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.966643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46260 (local address 127.17.16.67:42183)
0811 09:14:16.976556 (+ 9913us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:16.976565 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:16.983571 (+ 7006us) server_negotiation.cc:407] Connection header received
0811 09:14:16.983726 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:16.983735 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:16.984000 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:16.984212 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:16.985436 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.987329 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.989804 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.991345 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.991471 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.991694 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:16.992933 (+ 1239us) server_negotiation.cc:299] Negotiation successful
0811 09:14:16.993106 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9707,"thread_start_us":9471,"threads_started":1}
I20260811 09:14:16.995375 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.965506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:33047 (local address 127.0.0.1:56602)
0811 09:14:16.979508 (+ 14002us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:16.979557 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:16.979788 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:16.986987 (+ 7199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:16.986991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:16.987015 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:16.987448 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:16.987462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.989349 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.989355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:16.991451 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:16.991466 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.994713 (+ 3247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.994717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:16.994839 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.994861 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 09:14:16.994938 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 09:14:16.995065 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13760,"thread_start_us":13628,"threads_started":1}
I20260811 09:14:16.995638 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:16.965920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.17.16.65:33047)
0811 09:14:16.986242 (+ 20322us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:16.986250 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:16.986277 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:14:16.986369 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:16.986376 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:16.986563 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:16.986750 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:16.987626 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.989207 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.991678 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:16.994303 (+ 2625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:16.994433 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:16.994604 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:16.995028 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 09:14:16.995181 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20150,"thread_start_us":19961,"threads_started":1}
I20260811 09:14:16.996762 18836 tablet_service.cc:1467] Processing CreateTablet for tablet da87a1cba4594b1ea7451cb798f551f3 (DEFAULT_TABLE table=test-table [id=fd12af59ffe04656a64f689454242342]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:16.997010 19063 tablet_service.cc:1467] Processing CreateTablet for tablet da87a1cba4594b1ea7451cb798f551f3 (DEFAULT_TABLE table=test-table [id=fd12af59ffe04656a64f689454242342]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:16.998421 18836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da87a1cba4594b1ea7451cb798f551f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:16.998744 18622 tablet_service.cc:1467] Processing CreateTablet for tablet da87a1cba4594b1ea7451cb798f551f3 (DEFAULT_TABLE table=test-table [id=fd12af59ffe04656a64f689454242342]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:16.999830 18622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da87a1cba4594b1ea7451cb798f551f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:17.000684 19063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da87a1cba4594b1ea7451cb798f551f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:17.011842 19211 tablet_bootstrap.cc:492] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090: Bootstrap starting.
I20260811 09:14:17.015146 19213 tablet_bootstrap.cc:492] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c: Bootstrap starting.
I20260811 09:14:17.015194 19211 tablet_bootstrap.cc:654] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:17.016499 19211 log.cc:824] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:17.018929 19213 tablet_bootstrap.cc:654] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:17.020025 19211 tablet_bootstrap.cc:492] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090: No bootstrap required, opened a new log
I20260811 09:14:17.020164 19213 log.cc:824] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:17.020295 19211 ts_tablet_manager.cc:1397] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260811 09:14:17.021042 19212 tablet_bootstrap.cc:492] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab: Bootstrap starting.
I20260811 09:14:17.023566 19213 tablet_bootstrap.cc:492] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c: No bootstrap required, opened a new log
I20260811 09:14:17.023826 19213 ts_tablet_manager.cc:1397] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 09:14:17.023831 19212 tablet_bootstrap.cc:654] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:17.024770 19212 log.cc:824] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:17.027275 19212 tablet_bootstrap.cc:492] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab: No bootstrap required, opened a new log
I20260811 09:14:17.027505 19212 ts_tablet_manager.cc:1397] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 09:14:17.030755 19211 raft_consensus.cc:348] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407e3e411eed4013a87f3b2286118090" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33047 } } peers { permanent_uuid: "a7ffe18bd1594e478e73835e0aed507c" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42183 } } peers { permanent_uuid: "34e3e292e88c496eaeb5031f4bd042ab" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 41323 } }
I20260811 09:14:17.031468 19211 raft_consensus.cc:374] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:17.031603 19211 raft_consensus.cc:729] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407e3e411eed4013a87f3b2286118090, State: Initialized, Role: FOLLOWER
I20260811 09:14:17.032214 19211 consensus_queue.cc:260] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407e3e411eed4013a87f3b2286118090" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33047 } } peers { permanent_uuid: "a7ffe18bd1594e478e73835e0aed507c" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42183 } } peers { permanent_uuid: "34e3e292e88c496eaeb5031f4bd042ab" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 41323 } }
I20260811 09:14:17.033969 19211 ts_tablet_manager.cc:1428] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260811 09:14:17.034003 19213 raft_consensus.cc:348] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407e3e411eed4013a87f3b2286118090" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33047 } } peers { permanent_uuid: "a7ffe18bd1594e478e73835e0aed507c" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42183 } } peers { permanent_uuid: "34e3e292e88c496eaeb5031f4bd042ab" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 41323 } }
I20260811 09:14:17.034437 19213 raft_consensus.cc:374] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:17.034531 19213 raft_consensus.cc:729] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ffe18bd1594e478e73835e0aed507c, State: Initialized, Role: FOLLOWER
I20260811 09:14:17.035058 19213 consensus_queue.cc:260] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407e3e411eed4013a87f3b2286118090" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33047 } } peers { permanent_uuid: "a7ffe18bd1594e478e73835e0aed507c" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42183 } } peers { permanent_uuid: "34e3e292e88c496eaeb5031f4bd042ab" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 41323 } }
I20260811 09:14:17.036044 19182 heartbeater.cc:503] Master 127.17.16.126:46273 was elected leader, sending a full tablet report...
I20260811 09:14:17.036612 19213 ts_tablet_manager.cc:1428] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260811 09:14:17.036623 19212 raft_consensus.cc:348] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407e3e411eed4013a87f3b2286118090" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33047 } } peers { permanent_uuid: "a7ffe18bd1594e478e73835e0aed507c" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42183 } } peers { permanent_uuid: "34e3e292e88c496eaeb5031f4bd042ab" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 41323 } }
I20260811 09:14:17.037046 19212 raft_consensus.cc:374] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:17.037138 19212 raft_consensus.cc:729] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e3e292e88c496eaeb5031f4bd042ab, State: Initialized, Role: FOLLOWER
I20260811 09:14:17.037808 19212 consensus_queue.cc:260] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407e3e411eed4013a87f3b2286118090" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33047 } } peers { permanent_uuid: "a7ffe18bd1594e478e73835e0aed507c" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42183 } } peers { permanent_uuid: "34e3e292e88c496eaeb5031f4bd042ab" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 41323 } }
I20260811 09:14:17.038985 18953 heartbeater.cc:503] Master 127.17.16.126:46273 was elected leader, sending a full tablet report...
I20260811 09:14:17.040655 19212 ts_tablet_manager.cc:1428] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 09:14:17.062371 19221 raft_consensus.cc:482] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:17.062673 19221 raft_consensus.cc:504] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407e3e411eed4013a87f3b2286118090" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33047 } } peers { permanent_uuid: "a7ffe18bd1594e478e73835e0aed507c" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42183 } } peers { permanent_uuid: "34e3e292e88c496eaeb5031f4bd042ab" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 41323 } }
I20260811 09:14:17.064867 19221 leader_election.cc:288] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 407e3e411eed4013a87f3b2286118090 (127.17.16.65:33047), 34e3e292e88c496eaeb5031f4bd042ab (127.17.16.66:41323)
I20260811 09:14:17.071774 19210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:17.065548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:33865 (local address 127.17.16.66:41323)
0811 09:14:17.065749 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:17.065755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:17.065773 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:14:17.065971 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:17.065975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:17.066034 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:17.066098 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:17.066542 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.067599 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.069853 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.071001 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.071177 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:17.071253 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:17.071427 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 09:14:17.071575 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:14:17.071787 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:17.065275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:41323 (local address 127.17.16.67:33865)
0811 09:14:17.065622 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:17.065642 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:17.065792 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:17.066152 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:17.066156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:17.066191 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:17.066402 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:17.066413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.067740 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.067743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:17.069687 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:17.069703 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.071152 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.071156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:17.071285 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:17.071309 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:14:17.071377 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 09:14:17.071487 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214}
I20260811 09:14:17.072285 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:17.065378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:40393 (local address 127.17.16.65:33047)
0811 09:14:17.065587 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:17.065593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:17.066299 (+ 706us) server_negotiation.cc:407] Connection header received
0811 09:14:17.066452 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:17.066455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:17.066525 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:17.066616 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:17.067089 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.067982 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.070177 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.071358 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.071512 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:17.071578 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:17.071928 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 09:14:17.072060 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:14:17.072911 19224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:17.065305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:33047 (local address 127.17.16.67:40393)
0811 09:14:17.066190 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:17.066205 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:17.066340 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:17.066633 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:17.066637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:17.066657 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:17.066922 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:17.066929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.068118 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.068123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:17.070018 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:17.070035 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.071561 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.071568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:17.071734 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:17.071765 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:14:17.072441 (+ 676us) client_negotiation.cc:240] Negotiation successful
0811 09:14:17.072543 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":4,"thread_start_us":463,"threads_started":1}
I20260811 09:14:17.073290 18869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da87a1cba4594b1ea7451cb798f551f3" candidate_uuid: "a7ffe18bd1594e478e73835e0aed507c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e3e292e88c496eaeb5031f4bd042ab" is_pre_election: true
I20260811 09:14:17.073941 18650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da87a1cba4594b1ea7451cb798f551f3" candidate_uuid: "a7ffe18bd1594e478e73835e0aed507c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407e3e411eed4013a87f3b2286118090" is_pre_election: true
I20260811 09:14:17.074146 18869 raft_consensus.cc:2454] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7ffe18bd1594e478e73835e0aed507c in term 0.
I20260811 09:14:17.074441 18650 raft_consensus.cc:2454] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7ffe18bd1594e478e73835e0aed507c in term 0.
I20260811 09:14:17.075017 18994 leader_election.cc:302] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 34e3e292e88c496eaeb5031f4bd042ab, 407e3e411eed4013a87f3b2286118090, a7ffe18bd1594e478e73835e0aed507c; no voters:
I20260811 09:14:17.075394 19221 raft_consensus.cc:2781] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:17.075541 19221 raft_consensus.cc:482] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:17.075647 19221 raft_consensus.cc:3037] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:17.078310 19221 raft_consensus.cc:504] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407e3e411eed4013a87f3b2286118090" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33047 } } peers { permanent_uuid: "a7ffe18bd1594e478e73835e0aed507c" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42183 } } peers { permanent_uuid: "34e3e292e88c496eaeb5031f4bd042ab" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 41323 } }
I20260811 09:14:17.078922 19221 leader_election.cc:288] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [CANDIDATE]: Term 1 election: Requested vote from peers 407e3e411eed4013a87f3b2286118090 (127.17.16.65:33047), 34e3e292e88c496eaeb5031f4bd042ab (127.17.16.66:41323)
I20260811 09:14:17.079277 18650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da87a1cba4594b1ea7451cb798f551f3" candidate_uuid: "a7ffe18bd1594e478e73835e0aed507c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407e3e411eed4013a87f3b2286118090"
I20260811 09:14:17.079373 18869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da87a1cba4594b1ea7451cb798f551f3" candidate_uuid: "a7ffe18bd1594e478e73835e0aed507c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e3e292e88c496eaeb5031f4bd042ab"
I20260811 09:14:17.079485 18650 raft_consensus.cc:3037] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:17.079559 18869 raft_consensus.cc:3037] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:17.081828 18650 raft_consensus.cc:2454] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7ffe18bd1594e478e73835e0aed507c in term 1.
I20260811 09:14:17.081862 18869 raft_consensus.cc:2454] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7ffe18bd1594e478e73835e0aed507c in term 1.
I20260811 09:14:17.082180 18994 leader_election.cc:302] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e3e292e88c496eaeb5031f4bd042ab, a7ffe18bd1594e478e73835e0aed507c; no voters:
I20260811 09:14:17.082543 19221 raft_consensus.cc:2781] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:17.083084 19221 raft_consensus.cc:686] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [term 1 LEADER]: Becoming Leader. State: Replica: a7ffe18bd1594e478e73835e0aed507c, State: Running, Role: LEADER
I20260811 09:14:17.083525 19221 consensus_queue.cc:237] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407e3e411eed4013a87f3b2286118090" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33047 } } peers { permanent_uuid: "a7ffe18bd1594e478e73835e0aed507c" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42183 } } peers { permanent_uuid: "34e3e292e88c496eaeb5031f4bd042ab" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 41323 } }
I20260811 09:14:17.088671 18419 catalog_manager.cc:5495] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c reported cstate change: term changed from 0 to 1, leader changed from <none> to a7ffe18bd1594e478e73835e0aed507c (127.17.16.67). New cstate: current_term: 1 leader_uuid: "a7ffe18bd1594e478e73835e0aed507c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407e3e411eed4013a87f3b2286118090" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7ffe18bd1594e478e73835e0aed507c" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e3e292e88c496eaeb5031f4bd042ab" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 41323 } health_report { overall_health: UNKNOWN } } }
Aug 11 09:14:17 dist-test-slave-ncp9 krb5kdc[18346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:14:17.120296 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:17.104517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56618 (local address 127.17.16.65:33047)
0811 09:14:17.104684 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:17.104689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:17.105294 (+ 605us) server_negotiation.cc:407] Connection header received
0811 09:14:17.105478 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:17.105483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:17.105556 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:17.105663 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:17.107436 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.108391 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.109788 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.110239 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.110362 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:17.114478 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:17.114518 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:17.114539 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:17.114578 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:17.117628 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.118258 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.118266 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.118268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.118395 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.118750 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.118753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.118755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.119163 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:17.119484 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:17.119744 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 09:14:17.119946 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 09:14:17.134833 19187 tablet.cc:2369] T da87a1cba4594b1ea7451cb798f551f3 P a7ffe18bd1594e478e73835e0aed507c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 09:14:17 dist-test-slave-ncp9 krb5kdc[18346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439657, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260811 09:14:17.163815 18955 tablet.cc:2369] T da87a1cba4594b1ea7451cb798f551f3 P 34e3e292e88c496eaeb5031f4bd042ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 09:14:17 dist-test-slave-ncp9 krb5kdc[18346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439657, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
W20260811 09:14:17.171118 18729 tablet.cc:2369] T da87a1cba4594b1ea7451cb798f551f3 P 407e3e411eed4013a87f3b2286118090: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:14:17.174513 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:17.159987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56634 (local address 127.17.16.65:33047)
0811 09:14:17.160197 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:17.160204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:17.160564 (+ 360us) server_negotiation.cc:407] Connection header received
0811 09:14:17.160760 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:17.160764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:17.160832 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:17.160941 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:17.162413 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.163154 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.165130 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.165698 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.165816 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:17.169548 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:17.169554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:17.169557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:17.169591 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:17.172361 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.172932 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.172936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.172939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.173079 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.173392 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.173395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.173397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.173677 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:17.173914 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:17.174087 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 09:14:17.174249 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 09:14:17 dist-test-slave-ncp9 krb5kdc[18346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:14:17 dist-test-slave-ncp9 krb5kdc[18346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:14:17.221640 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:17.208778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.17.16.65:33047)
0811 09:14:17.209043 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:17.209050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:17.209290 (+ 240us) server_negotiation.cc:407] Connection header received
0811 09:14:17.209433 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:17.209436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:17.209502 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:17.209583 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:17.210841 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.211513 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.212846 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.213314 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.213405 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:17.217152 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:17.217157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:17.217160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:17.217203 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:17.219484 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.220062 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.220065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.220068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.220184 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.220444 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.220447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.220449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.220733 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:17.220952 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:17.221145 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 09:14:17.221325 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 09:14:17.223927 18591 tablet_service.cc:2908] Rejecting scan request for tablet da87a1cba4594b1ea7451cb798f551f3: Uninitialized: clean time has not yet been initialized
I20260811 09:14:17.227707 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 18513
I20260811 09:14:17.239408 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:17.228606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56662 (local address 127.17.16.65:33047)
0811 09:14:17.228750 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:17.228754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:17.229084 (+ 330us) server_negotiation.cc:407] Connection header received
0811 09:14:17.229213 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:17.229216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:17.229284 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:17.229358 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:17.230376 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.231025 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.232102 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.232669 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.232774 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:17.234765 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:17.234770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:17.234774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:17.234809 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:17.237468 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.238025 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.238028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.238030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.238133 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.238393 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.238396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.238399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.238650 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:17.238851 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:17.239018 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 09:14:17.239179 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 09:14:17.239987 18712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:17.327801 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 18513
I20260811 09:14:17.349431 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 18736
Aug 11 09:14:17 dist-test-slave-ncp9 krb5kdc[18346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:14:17.363400 19210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:17.350309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46950 (local address 127.17.16.66:41323)
0811 09:14:17.350485 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:17.350489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:17.350507 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:14:17.350565 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:17.350569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:17.350677 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:17.350758 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:17.351868 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.352615 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.353914 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.354309 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.354397 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:17.358271 (+ 3874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:17.358287 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:17.358304 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:17.358342 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:17.361115 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.361729 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.361737 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.361739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.361865 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.362144 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.362147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.362149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.362521 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:17.362773 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:17.362925 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 09:14:17.363082 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:14:17.364075 18945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:17.449918 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 18736
I20260811 09:14:17.476683 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 18964
Aug 11 09:14:17 dist-test-slave-ncp9 krb5kdc[18346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:14:17.491367 19209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:17.477459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.17.16.67:42183)
0811 09:14:17.477646 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:17.477651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:17.477670 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:14:17.477821 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:17.477825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:17.477886 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:17.477977 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:17.479160 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.479860 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.481224 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:17.481705 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:17.481820 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:17.485779 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:17.485796 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:17.485814 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:17.485850 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:17.488788 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.489487 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.489497 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.489499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.489655 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:17.489953 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:17.489956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:17.489958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:17.490347 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:17.490618 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:17.490797 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 09:14:17.491007 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:14:17.492060 19171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:17.581784 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 18964
I20260811 09:14:17.602677 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 18364
I20260811 09:14:17.603775 18460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:17.712333 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 18364
2026-08-11T09:14:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4200 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:14:17 dist-test-slave-ncp9 krb5kdc[19265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:14:17 dist-test-slave-ncp9 krb5kdc[19265](info): set up 2 sockets
Aug 11 09:14:17 dist-test-slave-ncp9 krb5kdc[19265](info): commencing operation
krb5kdc: starting...
W20260811 09:14:18.932834 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:14:19 dist-test-slave-ncp9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:14:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:14:19.122241 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:43959
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:43959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:19.439942 19289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:19.440220 19289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:19.440337 19289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:19.450006 19289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:19.450187 19289 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 09:14:19.450278 19289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:19.450395 19289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:19.450490 19289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:19.450600 19289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:19.461488 19289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:43959
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:43959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:19.462831 19289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:19.464186 19289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:19.475661 19300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:19.475620 19302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:19.475592 19305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:19.476742 19289 server_base.cc:1034] running on GCE node
I20260811 09:14:19.477504 19289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:19.478792 19289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:19.480010 19289 hybrid_clock.cc:648] HybridClock initialized: now 1786439659479954 us; error 26 us; skew 500 ppm
I20260811 09:14:19.480301 19289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:19 dist-test-slave-ncp9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439659, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:19.483276 19289 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:14:19.484689 19289 webserver.cc:460] Webserver started at http://127.17.16.126:43789/ using document root <none> and password file <none>
I20260811 09:14:19.485405 19289 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:19.485498 19289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:19.485786 19289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:19.488772 19289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "267f2c0cbc2b481aafb3b3e91c841cf3"
format_stamp: "Formatted at 2026-08-11 09:14:19 on dist-test-slave-ncp9"
server_key: "bc1b65a30470388b57db189f75ec9453"
server_key_iv: "4fb90c4d58925da04e4c1879b6fe1f98"
server_key_version: "encryptionkey@0"
I20260811 09:14:19.489588 19289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "267f2c0cbc2b481aafb3b3e91c841cf3"
format_stamp: "Formatted at 2026-08-11 09:14:19 on dist-test-slave-ncp9"
server_key: "bc1b65a30470388b57db189f75ec9453"
server_key_iv: "4fb90c4d58925da04e4c1879b6fe1f98"
server_key_version: "encryptionkey@0"
I20260811 09:14:19.494524 19289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:14:19.497920 19315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:19.499217 19289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:14:19.499366 19289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "267f2c0cbc2b481aafb3b3e91c841cf3"
format_stamp: "Formatted at 2026-08-11 09:14:19 on dist-test-slave-ncp9"
server_key: "bc1b65a30470388b57db189f75ec9453"
server_key_iv: "4fb90c4d58925da04e4c1879b6fe1f98"
server_key_version: "encryptionkey@0"
I20260811 09:14:19.499564 19289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:19.511191 19289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:19.514894 19289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:19.515141 19289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:19.535815 19289 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:43959
I20260811 09:14:19.535835 19432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:43959 every 8 connection(s)
I20260811 09:14:19.537245 19289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:19.539139 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 19289
I20260811 09:14:19.539373 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:19.540212 17473 external_mini_cluster.cc:1428] Setting key 96314f892e5a12a17df132b55fc6be79
I20260811 09:14:19.542016 19435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:14:19 dist-test-slave-ncp9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:19.556011 19435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3: Bootstrap starting.
I20260811 09:14:19.559427 19435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:19.560664 19435 log.cc:824] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:19.563151 19440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:19.542414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.17.16.126:43959)
0811 09:14:19.543144 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:19.543156 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:19.543199 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:14:19.544789 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:19.544811 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:19.545259 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:19.545589 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:19.547336 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:19.548705 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:19.550417 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:19.551095 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:19.551221 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:19.556048 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:19.556055 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:19.556071 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:19.556115 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:19.559096 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:19.559886 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:19.559893 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:19.559899 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:19.560072 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:19.560441 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:19.560444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:19.560446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:19.561045 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:19.561346 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:19.561792 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 09:14:19.562208 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":325,"threads_started":1}
I20260811 09:14:19.563656 19435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3: No bootstrap required, opened a new log
I20260811 09:14:19.571411 19435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267f2c0cbc2b481aafb3b3e91c841cf3" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43959 } }
I20260811 09:14:19.571796 19435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:19.571895 19435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267f2c0cbc2b481aafb3b3e91c841cf3, State: Initialized, Role: FOLLOWER
I20260811 09:14:19.572459 19435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267f2c0cbc2b481aafb3b3e91c841cf3" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43959 } }
I20260811 09:14:19.572681 19435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:19.572811 19435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:19.572974 19435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:19.575398 19435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267f2c0cbc2b481aafb3b3e91c841cf3" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43959 } }
I20260811 09:14:19.575906 19435 leader_election.cc:302] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 267f2c0cbc2b481aafb3b3e91c841cf3; no voters:
I20260811 09:14:19.576458 19435 leader_election.cc:288] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:19.576591 19442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:19.577307 19442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [term 1 LEADER]: Becoming Leader. State: Replica: 267f2c0cbc2b481aafb3b3e91c841cf3, State: Running, Role: LEADER
I20260811 09:14:19.577888 19442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267f2c0cbc2b481aafb3b3e91c841cf3" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43959 } }
I20260811 09:14:19.578527 19435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:19.580760 19445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 267f2c0cbc2b481aafb3b3e91c841cf3. Latest consensus state: current_term: 1 leader_uuid: "267f2c0cbc2b481aafb3b3e91c841cf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267f2c0cbc2b481aafb3b3e91c841cf3" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43959 } } }
I20260811 09:14:19.581199 19445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:19.581570 19444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "267f2c0cbc2b481aafb3b3e91c841cf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267f2c0cbc2b481aafb3b3e91c841cf3" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43959 } } }
I20260811 09:14:19.581854 19444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:19.583124 19450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:19.588197 19450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:19.595968 19450 catalog_manager.cc:1347] Generated new cluster ID: 9fc1c53cd5e54ac89a0df5b30d87a08d
I20260811 09:14:19.596089 19450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:19.625128 19450 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:19.627135 19450 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:19.649164 19450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3: Generated new TSK 0
I20260811 09:14:19.650054 19450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:14:19.702068 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:43959
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:20.027709 19470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:20.028033 19470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:20.028178 19470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:20.035439 19470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:20.035570 19470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:20.035698 19470 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:14:20.035808 19470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:14:20.045480 19470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.16.126:43959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:20.046741 19470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:20.048030 19470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:20.060745 19486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:20.060702 19480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:20.060777 19483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:20.061511 19470 server_base.cc:1034] running on GCE node
I20260811 09:14:20.062105 19470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:20.063059 19470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:20.064271 19470 hybrid_clock.cc:648] HybridClock initialized: now 1786439660064213 us; error 30 us; skew 500 ppm
I20260811 09:14:20.064574 19470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:20 dist-test-slave-ncp9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:20.067674 19470 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:14:20.069160 19470 webserver.cc:460] Webserver started at http://127.17.16.65:35649/ using document root <none> and password file <none>
I20260811 09:14:20.069830 19470 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:20.069931 19470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:20.070259 19470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:20.073280 19470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "a0cd4f542eac492ab84dbb43327ef8ad"
format_stamp: "Formatted at 2026-08-11 09:14:20 on dist-test-slave-ncp9"
server_key: "02372ffcd009b8c1a1b443bd930154ae"
server_key_iv: "e1957e816e3c3c8f2d03492d975062fa"
server_key_version: "encryptionkey@0"
I20260811 09:14:20.074056 19470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "a0cd4f542eac492ab84dbb43327ef8ad"
format_stamp: "Formatted at 2026-08-11 09:14:20 on dist-test-slave-ncp9"
server_key: "02372ffcd009b8c1a1b443bd930154ae"
server_key_iv: "e1957e816e3c3c8f2d03492d975062fa"
server_key_version: "encryptionkey@0"
I20260811 09:14:20.079239 19470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:14:20.082701 19499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:20.084052 19470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:14:20.084193 19470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "a0cd4f542eac492ab84dbb43327ef8ad"
format_stamp: "Formatted at 2026-08-11 09:14:20 on dist-test-slave-ncp9"
server_key: "02372ffcd009b8c1a1b443bd930154ae"
server_key_iv: "e1957e816e3c3c8f2d03492d975062fa"
server_key_version: "encryptionkey@0"
I20260811 09:14:20.084367 19470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:20.100620 19470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:20.104701 19470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:20.104969 19470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:20.107702 19470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:20.107805 19470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:20.107945 19470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:20.108016 19470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:14:20 dist-test-slave-ncp9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:20.131518 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:20.108371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:47517 (local address 127.17.16.126:43959)
0811 09:14:20.108917 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:20.108925 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:20.110081 (+ 1156us) server_negotiation.cc:407] Connection header received
0811 09:14:20.112392 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:20.112399 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:20.112492 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:20.112597 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:20.115022 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.116096 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.119050 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.119523 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.119638 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:20.124099 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:20.124105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:20.124109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:20.124149 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:20.127531 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:20.128673 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:20.128680 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:20.128682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:20.128823 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:20.129452 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:20.129458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:20.129462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:20.129899 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:20.130066 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:20.130820 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 09:14:20.131087 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":288,"threads_started":1}
I20260811 09:14:20.132508 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:20.109071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:43959 (local address 127.17.16.65:47517)
0811 09:14:20.109911 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:20.109957 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:20.112036 (+ 2079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:20.112950 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:20.112963 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:20.113824 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:20.114804 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:20.114838 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.116266 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.116273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:20.118871 (+ 2598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:20.118890 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.119778 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.119785 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:20.120066 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:20.120817 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:20.120845 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:20.123817 (+ 2972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:20.127699 (+ 3882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:20.127711 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:20.127735 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:20.128522 (+ 787us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:20.128973 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:20.128979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:20.128984 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:20.129294 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:20.130053 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:20.130062 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:20.130404 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 09:14:20.130673 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 09:14:20.131187 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":352,"threads_started":1}
I20260811 09:14:20.152222 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:20.139177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:42757 (local address 127.17.16.126:43959)
0811 09:14:20.139359 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:20.139365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:20.140053 (+ 688us) server_negotiation.cc:407] Connection header received
0811 09:14:20.140201 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:20.140206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:20.140288 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:20.140386 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:20.141549 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.142743 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.144126 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.144807 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.144967 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:20.146619 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:20.146626 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:20.146632 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:20.146675 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:20.149726 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:20.150366 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:20.150372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:20.150374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:20.150513 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:20.150860 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:20.150864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:20.150866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:20.151128 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:20.151258 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:20.151676 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 09:14:20.151885 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:14:20.152539 19631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:20.139300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:43959 (local address 127.17.16.65:42757)
0811 09:14:20.139957 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:20.139978 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:20.140098 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:20.140417 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:20.140420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:20.140941 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:20.141365 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:20.141386 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.142916 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.142920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:20.143999 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:20.144013 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.145046 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.145051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:20.145191 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:20.145778 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:20.145795 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:20.146463 (+ 668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:20.149851 (+ 3388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:20.149856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:20.149861 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:20.150255 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:20.150626 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:20.150631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:20.150635 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:20.150759 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:20.151238 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:20.151243 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:20.151431 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 09:14:20.151562 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 09:14:20.151914 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":351,"threads_started":1}
I20260811 09:14:20.166704 19470 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:42203
I20260811 09:14:20.166733 19721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:42203 every 8 connection(s)
I20260811 09:14:20.167954 19470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:14:20.170084 19724 heartbeater.cc:347] Connected to a master server at 127.17.16.126:43959
I20260811 09:14:20.170332 19724 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:20.171036 19724 heartbeater.cc:511] Master 127.17.16.126:43959 requested a full tablet report, sending...
I20260811 09:14:20.173249 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 19470
I20260811 09:14:20.173339 19353 ts_manager.cc:194] Registered new tserver with Master: a0cd4f542eac492ab84dbb43327ef8ad (127.17.16.65:42203)
I20260811 09:14:20.173499 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:14:20.174410 17473 external_mini_cluster.cc:1428] Setting key 281d05d6fa2392eb8b9e6997b92b7e84
I20260811 09:14:20.175403 19353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:47517
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:14:20.218251 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:43959
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:20.544407 19730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:20.544771 19730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:20.544955 19730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:20.556236 19730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:20.556372 19730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:20.556509 19730 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:14:20.556622 19730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:14:20.566967 19730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.16.126:43959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:20.568194 19730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:20.569517 19730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:20.582796 19736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:20.582799 19735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:20.583048 19739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:20.583591 19730 server_base.cc:1034] running on GCE node
I20260811 09:14:20.584227 19730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:20.585215 19730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:20.586427 19730 hybrid_clock.cc:648] HybridClock initialized: now 1786439660586357 us; error 46 us; skew 500 ppm
I20260811 09:14:20.586735 19730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:20 dist-test-slave-ncp9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:20.589918 19730 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:14:20.591277 19730 webserver.cc:460] Webserver started at http://127.17.16.66:36829/ using document root <none> and password file <none>
I20260811 09:14:20.591950 19730 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:20.592053 19730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:20.592361 19730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:20.595227 19730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "dcef4046fd5848f3a2f63470bf4a4158"
format_stamp: "Formatted at 2026-08-11 09:14:20 on dist-test-slave-ncp9"
server_key: "8cfced82a7ff7698f8eca9814feb2d47"
server_key_iv: "bc73bd7bc165ab449b1b44d477b88533"
server_key_version: "encryptionkey@0"
I20260811 09:14:20.595988 19730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "dcef4046fd5848f3a2f63470bf4a4158"
format_stamp: "Formatted at 2026-08-11 09:14:20 on dist-test-slave-ncp9"
server_key: "8cfced82a7ff7698f8eca9814feb2d47"
server_key_iv: "bc73bd7bc165ab449b1b44d477b88533"
server_key_version: "encryptionkey@0"
I20260811 09:14:20.600944 19730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 09:14:20.604157 19749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:20.605440 19730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:14:20.605595 19730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "dcef4046fd5848f3a2f63470bf4a4158"
format_stamp: "Formatted at 2026-08-11 09:14:20 on dist-test-slave-ncp9"
server_key: "8cfced82a7ff7698f8eca9814feb2d47"
server_key_iv: "bc73bd7bc165ab449b1b44d477b88533"
server_key_version: "encryptionkey@0"
I20260811 09:14:20.605793 19730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:20.621287 19730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:20.625131 19730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:20.625391 19730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:20.628062 19730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:20.628155 19730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:20.628298 19730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:20.628362 19730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:14:20 dist-test-slave-ncp9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439660, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:20.652721 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:20.629313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:43959 (local address 127.17.16.66:58235)
0811 09:14:20.630209 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:20.630253 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:20.632233 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:20.633054 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:20.633066 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:20.633901 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:20.634927 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:20.634956 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.636345 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.636351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:20.638423 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:20.638440 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.639394 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.639402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:20.639718 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:20.640574 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:20.640654 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:20.644224 (+ 3570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:20.647825 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:20.647834 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:20.647850 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:20.648459 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:20.648888 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:20.648913 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:20.648917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:20.649142 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:20.649975 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:20.649981 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:20.650437 (+ 456us) client_negotiation.cc:769] Sending connection context
0811 09:14:20.650715 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 09:14:20.651221 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":362,"threads_started":1}
I20260811 09:14:20.653898 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:20.628731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:58235 (local address 127.17.16.126:43959)
0811 09:14:20.628963 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:20.628968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:20.630403 (+ 1435us) server_negotiation.cc:407] Connection header received
0811 09:14:20.632518 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:20.632534 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:20.632627 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:20.632728 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:20.635144 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.636194 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.638570 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.639154 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.639268 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:20.644493 (+ 5225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:20.644500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:20.644505 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:20.644550 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:20.647655 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:20.648623 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:20.648629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:20.648632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:20.648773 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:20.649311 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:20.649316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:20.649319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:20.649749 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:20.649907 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:20.653474 (+ 3567us) server_negotiation.cc:299] Negotiation successful
0811 09:14:20.653661 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:14:20.673496 19912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:20.659430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:43959 (local address 127.17.16.66:42659)
0811 09:14:20.659988 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:20.660003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:20.660133 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:20.660412 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:20.660417 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:20.660820 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:20.661247 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:20.661259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.662541 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.662546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:20.663914 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:20.663927 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.664874 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.664880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:20.665102 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:20.665723 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:20.665747 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:20.666874 (+ 1127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:20.670592 (+ 3718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:20.670598 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:20.670602 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:20.671085 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:20.671532 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:20.671541 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:20.671545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:20.671689 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:20.672243 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:20.672248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:20.672440 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 09:14:20.672612 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 09:14:20.672966 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":301,"threads_started":1}
I20260811 09:14:20.673594 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:20.659335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:42659 (local address 127.17.16.126:43959)
0811 09:14:20.659484 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:20.659488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:20.660089 (+ 601us) server_negotiation.cc:407] Connection header received
0811 09:14:20.660234 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:20.660240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:20.660300 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:20.660375 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:20.661432 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.662340 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.664061 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:20.664541 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:20.664659 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:20.667078 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:20.667085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:20.667091 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:20.667134 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:20.670433 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:20.671220 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:20.671226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:20.671229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:20.671393 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:20.671796 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:20.671802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:20.671806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:20.672129 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:20.672261 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:20.673075 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 09:14:20.673313 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 09:14:20.674103 19730 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:44969
I20260811 09:14:20.674103 19973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:44969 every 8 connection(s)
I20260811 09:14:20.675426 19730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:14:20.677532 19974 heartbeater.cc:347] Connected to a master server at 127.17.16.126:43959
I20260811 09:14:20.677732 19974 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:20.678216 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 19730
I20260811 09:14:20.678427 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:14:20.678458 19974 heartbeater.cc:511] Master 127.17.16.126:43959 requested a full tablet report, sending...
I20260811 09:14:20.678995 17473 external_mini_cluster.cc:1428] Setting key a6d6c7a88dd55cb2d2c683ab65c1076d
I20260811 09:14:20.680085 19350 ts_manager.cc:194] Registered new tserver with Master: dcef4046fd5848f3a2f63470bf4a4158 (127.17.16.66:44969)
I20260811 09:14:20.681102 19350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:58235
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:14:20.723501 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:43959
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:21.047053 19983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:21.047336 19983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:21.047477 19983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:21.054461 19983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:21.054570 19983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:21.054695 19983 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:14:21.054808 19983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:14:21.064459 19983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.16.126:43959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:21.065734 19983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:21.067050 19983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:21.080189 19991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:21.080272 19989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:21.080166 19994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:21.082527 19983 server_base.cc:1034] running on GCE node
I20260811 09:14:21.083120 19983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:21.084059 19983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:21.085268 19983 hybrid_clock.cc:648] HybridClock initialized: now 1786439661085207 us; error 36 us; skew 500 ppm
I20260811 09:14:21.085592 19983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:21 dist-test-slave-ncp9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439661, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:21.088608 19983 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:14:21.090108 19983 webserver.cc:460] Webserver started at http://127.17.16.67:44069/ using document root <none> and password file <none>
I20260811 09:14:21.090809 19983 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:21.090909 19983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:21.091214 19983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:21.094142 19983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "80f826ed499b4c23a3390b1f7b645f36"
format_stamp: "Formatted at 2026-08-11 09:14:21 on dist-test-slave-ncp9"
server_key: "20680ef540a574902e8b3249d820b4ec"
server_key_iv: "9dba6a312e31cdfc813e338520140cf1"
server_key_version: "encryptionkey@0"
I20260811 09:14:21.094903 19983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "80f826ed499b4c23a3390b1f7b645f36"
format_stamp: "Formatted at 2026-08-11 09:14:21 on dist-test-slave-ncp9"
server_key: "20680ef540a574902e8b3249d820b4ec"
server_key_iv: "9dba6a312e31cdfc813e338520140cf1"
server_key_version: "encryptionkey@0"
I20260811 09:14:21.099882 19983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:14:21.103255 20002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:21.104609 19983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:14:21.104776 19983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "80f826ed499b4c23a3390b1f7b645f36"
format_stamp: "Formatted at 2026-08-11 09:14:21 on dist-test-slave-ncp9"
server_key: "20680ef540a574902e8b3249d820b4ec"
server_key_iv: "9dba6a312e31cdfc813e338520140cf1"
server_key_version: "encryptionkey@0"
I20260811 09:14:21.105011 19983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:21.124125 19983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:21.127945 19983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:21.128190 19983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:21.130856 19983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:21.130963 19983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:21.131112 19983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:21.131220 19983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 09:14:21 dist-test-slave-ncp9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439661, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:21.154518 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.131294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:46797 (local address 127.17.16.126:43959)
0811 09:14:21.131460 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.131467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.132935 (+ 1468us) server_negotiation.cc:407] Connection header received
0811 09:14:21.135113 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.135119 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.135202 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.135292 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:21.136775 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.137809 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.140679 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.141296 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.141443 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.146544 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:21.146552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:21.146558 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:21.146603 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:21.150263 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:21.151336 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:21.151342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:21.151347 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:21.151532 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:21.152158 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:21.152164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:21.152167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:21.152534 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:21.152676 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.153859 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.154115 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 09:14:21.156487 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.131843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:43959 (local address 127.17.16.67:46797)
0811 09:14:21.132750 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.132804 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.134774 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.135561 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.135568 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.136043 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:21.136590 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.136608 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.138013 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.138022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.140502 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.140524 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.141593 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.141601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.141948 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.142644 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:21.142683 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:21.146257 (+ 3574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:21.150421 (+ 4164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:21.150431 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:21.150452 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:21.151197 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:21.151707 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:21.151716 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:21.151720 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:21.152031 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:21.152705 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:21.152719 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:21.153340 (+ 621us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.153735 (+ 395us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.154383 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":341,"threads_started":1}
I20260811 09:14:21.172091 19983 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:43331
I20260811 09:14:21.172132 20220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:43331 every 8 connection(s)
I20260811 09:14:21.173404 19983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:14:21.176910 20221 heartbeater.cc:347] Connected to a master server at 127.17.16.126:43959
I20260811 09:14:21.177175 20221 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:21.177965 20221 heartbeater.cc:511] Master 127.17.16.126:43959 requested a full tablet report, sending...
I20260811 09:14:21.178351 19724 heartbeater.cc:503] Master 127.17.16.126:43959 was elected leader, sending a full tablet report...
I20260811 09:14:21.178658 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.163958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:37419 (local address 127.17.16.126:43959)
0811 09:14:21.164171 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.164178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.164906 (+ 728us) server_negotiation.cc:407] Connection header received
0811 09:14:21.165169 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.165174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.165253 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.165347 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:21.166681 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.167709 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.169556 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.170201 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.170291 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.172746 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:21.172750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:21.172754 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:21.172783 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:21.175505 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:21.176353 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:21.176359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:21.176364 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:21.176545 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:21.177009 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:21.177014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:21.177017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:21.177371 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:21.177555 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.178093 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.178327 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:14:21.178719 20193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.163985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:43959 (local address 127.17.16.67:37419)
0811 09:14:21.164766 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.164796 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.164971 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.165379 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.165383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.166033 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:21.166509 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.166526 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.167873 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.167878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.169397 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.169412 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.170608 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.170614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.170791 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.171456 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:21.171483 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:21.172571 (+ 1088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:21.175639 (+ 3068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:21.175644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:21.175647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:21.176205 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:21.176665 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:21.176670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:21.176673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:21.176830 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:21.177647 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:21.177652 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:21.177822 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.177936 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.178253 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":433,"threads_started":1}
I20260811 09:14:21.180437 19353 ts_manager.cc:194] Registered new tserver with Master: 80f826ed499b4c23a3390b1f7b645f36 (127.17.16.67:43331)
I20260811 09:14:21.181211 19353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:46797
I20260811 09:14:21.181878 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 19983
I20260811 09:14:21.182060 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:14:21.182655 17473 external_mini_cluster.cc:1428] Setting key 0a4224df6a8f5eba04a11863f20a9ec6
I20260811 09:14:21.185612 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:14:21 dist-test-slave-ncp9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439661, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:14:21 dist-test-slave-ncp9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439661, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:21.240357 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.220771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.17.16.126:43959)
0811 09:14:21.220982 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.220987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.221237 (+ 250us) server_negotiation.cc:407] Connection header received
0811 09:14:21.221418 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.221422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.221492 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.221569 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:21.222896 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.223634 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.225161 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.225568 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.225662 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.235119 (+ 9457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:21.235125 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:21.235128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:21.235176 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:21.237731 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:21.238389 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:21.238393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:21.238395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:21.238534 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:21.238924 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:21.238927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:21.238929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:21.239233 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:21.239467 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.239771 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.240000 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:14:21.244482 19353 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:14:21.251397 19353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:21.275393 20251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.265127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:43331 (local address 127.0.0.1:48730)
0811 09:14:21.266964 (+ 1837us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.266977 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.267088 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.267912 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.267920 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.267952 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:21.268529 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.268547 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.270684 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.270690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.272970 (+ 2280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.272980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.274416 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.274423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.274600 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.274651 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.274792 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.274975 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1755,"mutex_wait_us":79,"thread_start_us":1646,"threads_started":1}
I20260811 09:14:21.275604 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.263953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.17.16.65:42203)
0811 09:14:21.267893 (+ 3940us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.267901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.267926 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:14:21.268024 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.268032 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.268245 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.269276 (+ 1031us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:21.269479 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.270838 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.273138 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.274558 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.274731 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.274917 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.275152 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.275313 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3805,"thread_start_us":3673,"threads_started":1}
I20260811 09:14:21.276260 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.265795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48730 (local address 127.17.16.67:43331)
0811 09:14:21.266619 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.266625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.267083 (+ 458us) server_negotiation.cc:407] Connection header received
0811 09:14:21.267233 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.267243 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.267501 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.267704 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:21.268825 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.270546 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.273131 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.274244 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.274371 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.274619 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.275825 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.275950 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":512,"threads_started":1}
I20260811 09:14:21.275550 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.263593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42203 (local address 127.0.0.1:55234)
0811 09:14:21.264343 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.264383 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.264562 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.268472 (+ 3910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.268476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.268509 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:21.269049 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.269063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.270978 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.270983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.272960 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.272973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.274795 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.274802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.274971 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.275000 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.275086 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.275212 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":30,"thread_start_us":482,"threads_started":1}
I20260811 09:14:21.277155 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.263943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.17.16.66:44969)
0811 09:14:21.267550 (+ 3607us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.267558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.267584 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:14:21.267676 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.267684 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.267893 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.268090 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:21.269116 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.270191 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.273104 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.275330 (+ 2226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.276400 (+ 1070us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.276609 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.276685 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.276848 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3452,"thread_start_us":3336,"threads_started":1}
I20260811 09:14:21.276278 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.263677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:44969 (local address 127.0.0.1:40424)
0811 09:14:21.265198 (+ 1521us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.265217 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.265328 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.268103 (+ 2775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.268108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.268159 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:21.268724 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.268737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.270313 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.270319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.272931 (+ 2612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.272948 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.275591 (+ 2643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.275598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.275763 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.275796 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.275862 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.275954 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1389,"mutex_wait_us":92,"spinlock_wait_cycles":1792,"thread_start_us":1229,"threads_started":1}
I20260811 09:14:21.280300 19602 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7ee59d8c694248a831b8cd529fc161 (DEFAULT_TABLE table=test-table [id=c20387bb71ac4c2ba6c5ac60cab0b7fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:21.280378 20089 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7ee59d8c694248a831b8cd529fc161 (DEFAULT_TABLE table=test-table [id=c20387bb71ac4c2ba6c5ac60cab0b7fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:21.281697 19602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7ee59d8c694248a831b8cd529fc161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:21.281530 19836 tablet_service.cc:1467] Processing CreateTablet for tablet 4b7ee59d8c694248a831b8cd529fc161 (DEFAULT_TABLE table=test-table [id=c20387bb71ac4c2ba6c5ac60cab0b7fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:21.282160 20089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7ee59d8c694248a831b8cd529fc161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:21.282826 19836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b7ee59d8c694248a831b8cd529fc161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:21.292878 20253 tablet_bootstrap.cc:492] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: Bootstrap starting.
I20260811 09:14:21.296267 20254 tablet_bootstrap.cc:492] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: Bootstrap starting.
I20260811 09:14:21.297201 20253 tablet_bootstrap.cc:654] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:21.298507 20253 log.cc:824] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:21.300148 20254 tablet_bootstrap.cc:654] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:21.301606 20254 log.cc:824] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:21.302765 20253 tablet_bootstrap.cc:492] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: No bootstrap required, opened a new log
I20260811 09:14:21.303040 20253 ts_tablet_manager.cc:1397] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 09:14:21.304291 20254 tablet_bootstrap.cc:492] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: No bootstrap required, opened a new log
I20260811 09:14:21.304527 20254 ts_tablet_manager.cc:1397] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 09:14:21.300696 20255 tablet_bootstrap.cc:492] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: Bootstrap starting.
I20260811 09:14:21.311472 20253 raft_consensus.cc:348] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.311944 20253 raft_consensus.cc:374] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:21.312031 20255 tablet_bootstrap.cc:654] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:21.312071 20253 raft_consensus.cc:729] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0cd4f542eac492ab84dbb43327ef8ad, State: Initialized, Role: FOLLOWER
I20260811 09:14:21.311842 20254 raft_consensus.cc:348] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.312233 20254 raft_consensus.cc:374] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:21.312327 20254 raft_consensus.cc:729] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80f826ed499b4c23a3390b1f7b645f36, State: Initialized, Role: FOLLOWER
I20260811 09:14:21.312846 20254 consensus_queue.cc:260] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.312844 20253 consensus_queue.cc:260] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.313843 20255 log.cc:824] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:21.314656 20254 ts_tablet_manager.cc:1428] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260811 09:14:21.315347 20221 heartbeater.cc:503] Master 127.17.16.126:43959 was elected leader, sending a full tablet report...
I20260811 09:14:21.321053 20253 ts_tablet_manager.cc:1428] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: Time spent starting tablet: real 0.018s user 0.014s sys 0.005s
I20260811 09:14:21.323709 20255 tablet_bootstrap.cc:492] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: No bootstrap required, opened a new log
I20260811 09:14:21.324018 20255 ts_tablet_manager.cc:1397] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.009s
I20260811 09:14:21.331936 20255 raft_consensus.cc:348] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.332290 20255 raft_consensus.cc:374] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:21.332369 20255 raft_consensus.cc:729] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcef4046fd5848f3a2f63470bf4a4158, State: Initialized, Role: FOLLOWER
I20260811 09:14:21.332960 20255 consensus_queue.cc:260] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.333963 19974 heartbeater.cc:503] Master 127.17.16.126:43959 was elected leader, sending a full tablet report...
I20260811 09:14:21.334499 20255 ts_tablet_manager.cc:1428] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: Time spent starting tablet: real 0.010s user 0.007s sys 0.005s
I20260811 09:14:21.418406 20262 raft_consensus.cc:482] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:21.418732 20262 raft_consensus.cc:504] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.420601 20262 leader_election.cc:288] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0cd4f542eac492ab84dbb43327ef8ad (127.17.16.65:42203), dcef4046fd5848f3a2f63470bf4a4158 (127.17.16.66:44969)
W20260811 09:14:21.420861 19725 tablet.cc:2369] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:14:21.427367 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.421724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:34539 (local address 127.17.16.65:42203)
0811 09:14:21.421896 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.421900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.421915 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:14:21.421983 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.421986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.422042 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.422122 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:21.422596 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.423474 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.425439 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.426474 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.426683 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.426764 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.426953 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.427082 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:14:21.427377 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.421063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42203 (local address 127.17.16.67:34539)
0811 09:14:21.421358 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.421377 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.421504 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.422186 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.422204 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.422240 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:21.422442 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.422450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.423612 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.423616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.425284 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.425299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.426638 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.426643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.426791 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.426829 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.426913 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.427026 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
W20260811 09:14:21.427623 20222 tablet.cc:2369] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:14:21.427934 19975 tablet.cc:2369] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:14:21.428702 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.421117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:34115 (local address 127.17.16.66:44969)
0811 09:14:21.421581 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.421587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.422596 (+ 1009us) server_negotiation.cc:407] Connection header received
0811 09:14:21.422751 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.422756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.422835 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.422928 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:21.423514 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.424253 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.426241 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.427357 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.427515 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.427584 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.428290 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.428435 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316}
I20260811 09:14:21.429255 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.421343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:44969 (local address 127.17.16.67:34115)
0811 09:14:21.422494 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.422509 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.422623 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.423025 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.423029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.423052 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:21.423345 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.423352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.424400 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.424405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.426078 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.426090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.427887 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.427893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.428082 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.428116 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.428795 (+ 679us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.429021 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"thread_start_us":817,"threads_started":1}
I20260811 09:14:21.429114 19643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7ee59d8c694248a831b8cd529fc161" candidate_uuid: "80f826ed499b4c23a3390b1f7b645f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" is_pre_election: true
I20260811 09:14:21.429646 19643 raft_consensus.cc:2454] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80f826ed499b4c23a3390b1f7b645f36 in term 0.
I20260811 09:14:21.430388 20004 leader_election.cc:302] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80f826ed499b4c23a3390b1f7b645f36, a0cd4f542eac492ab84dbb43327ef8ad; no voters:
I20260811 09:14:21.430286 19872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7ee59d8c694248a831b8cd529fc161" candidate_uuid: "80f826ed499b4c23a3390b1f7b645f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcef4046fd5848f3a2f63470bf4a4158" is_pre_election: true
I20260811 09:14:21.430827 20262 raft_consensus.cc:2781] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:21.430860 19872 raft_consensus.cc:2454] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80f826ed499b4c23a3390b1f7b645f36 in term 0.
I20260811 09:14:21.430967 20262 raft_consensus.cc:482] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:21.431077 20262 raft_consensus.cc:3037] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:21.433619 20262 raft_consensus.cc:504] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.434201 20262 leader_election.cc:288] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [CANDIDATE]: Term 1 election: Requested vote from peers a0cd4f542eac492ab84dbb43327ef8ad (127.17.16.65:42203), dcef4046fd5848f3a2f63470bf4a4158 (127.17.16.66:44969)
I20260811 09:14:21.434682 19643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7ee59d8c694248a831b8cd529fc161" candidate_uuid: "80f826ed499b4c23a3390b1f7b645f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0cd4f542eac492ab84dbb43327ef8ad"
I20260811 09:14:21.434679 19872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b7ee59d8c694248a831b8cd529fc161" candidate_uuid: "80f826ed499b4c23a3390b1f7b645f36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcef4046fd5848f3a2f63470bf4a4158"
I20260811 09:14:21.434875 19872 raft_consensus.cc:3037] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:21.434867 19643 raft_consensus.cc:3037] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:21.437338 19643 raft_consensus.cc:2454] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80f826ed499b4c23a3390b1f7b645f36 in term 1.
I20260811 09:14:21.437371 19872 raft_consensus.cc:2454] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80f826ed499b4c23a3390b1f7b645f36 in term 1.
I20260811 09:14:21.437717 20004 leader_election.cc:302] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80f826ed499b4c23a3390b1f7b645f36, a0cd4f542eac492ab84dbb43327ef8ad; no voters:
I20260811 09:14:21.437986 20262 raft_consensus.cc:2781] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:21.438452 20262 raft_consensus.cc:686] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 1 LEADER]: Becoming Leader. State: Replica: 80f826ed499b4c23a3390b1f7b645f36, State: Running, Role: LEADER
I20260811 09:14:21.438962 20262 consensus_queue.cc:237] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.443570 19353 catalog_manager.cc:5495] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80f826ed499b4c23a3390b1f7b645f36 (127.17.16.67). New cstate: current_term: 1 leader_uuid: "80f826ed499b4c23a3390b1f7b645f36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } health_report { overall_health: HEALTHY } } }
I20260811 09:14:21.479331 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.473638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48732 (local address 127.17.16.67:43331)
0811 09:14:21.473863 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.473869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.474011 (+ 142us) server_negotiation.cc:407] Connection header received
0811 09:14:21.474380 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.474384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.474476 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.474608 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:21.475014 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.475749 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.477175 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.477589 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.477683 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.478127 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:21.478778 (+ 651us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:21.478891 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.478981 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.479053 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:14:21.489130 19643 raft_consensus.cc:1261] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad [term 1 FOLLOWER]: Refusing update from remote peer 80f826ed499b4c23a3390b1f7b645f36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:21.490289 19872 raft_consensus.cc:1261] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158 [term 1 FOLLOWER]: Refusing update from remote peer 80f826ed499b4c23a3390b1f7b645f36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:21.490346 20268 consensus_queue.cc:1035] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:21.491159 20262 consensus_queue.cc:1035] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:21.526607 19353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:21.526937 19353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:21.530825 19353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3: Sending DeleteTablet for 3 replicas of tablet 4b7ee59d8c694248a831b8cd529fc161
I20260811 09:14:21.532186 19836 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7ee59d8c694248a831b8cd529fc161 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:21 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:40424
I20260811 09:14:21.533854 20089 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7ee59d8c694248a831b8cd529fc161 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:21 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:48730
I20260811 09:14:21.534441 19602 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b7ee59d8c694248a831b8cd529fc161 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:21 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:55234
I20260811 09:14:21.542515 20293 tablet_replica.cc:330] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: stopping tablet replica
I20260811 09:14:21.542749 20290 tablet_replica.cc:330] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: stopping tablet replica
I20260811 09:14:21.543200 20293 raft_consensus.cc:2229] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:21.543305 20290 raft_consensus.cc:2229] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:14:21.543850 20293 raft_consensus.cc:2258] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:21.544273 20290 raft_consensus.cc:2258] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:21.546800 20290 ts_tablet_manager.cc:1905] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:21.547391 19350 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:14:21.548146 19350 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:21.548329 20293 ts_tablet_manager.cc:1905] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:21.554849 20290 ts_tablet_manager.cc:1918] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:14:21.555202 20290 log.cc:1196] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/4b7ee59d8c694248a831b8cd529fc161
I20260811 09:14:21.556035 20290 ts_tablet_manager.cc:1939] T 4b7ee59d8c694248a831b8cd529fc161 P 80f826ed499b4c23a3390b1f7b645f36: Deleting consensus metadata
I20260811 09:14:21.557276 19602 tablet_service.cc:1467] Processing CreateTablet for tablet 9081a6162cc8470895568512f5a95d79 (DEFAULT_TABLE table=test-table [id=80bae4aa8446414a8cdbce0ec486932b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:21.557736 19602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9081a6162cc8470895568512f5a95d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:21.557930 19834 tablet_service.cc:1467] Processing CreateTablet for tablet 9081a6162cc8470895568512f5a95d79 (DEFAULT_TABLE table=test-table [id=80bae4aa8446414a8cdbce0ec486932b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:21.558581 19834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9081a6162cc8470895568512f5a95d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:21.562678 20089 tablet_service.cc:1467] Processing CreateTablet for tablet 9081a6162cc8470895568512f5a95d79 (DEFAULT_TABLE table=test-table [id=80bae4aa8446414a8cdbce0ec486932b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:21.563145 20089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9081a6162cc8470895568512f5a95d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:21.563158 20293 ts_tablet_manager.cc:1918] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:14:21.563341 20293 log.cc:1196] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/4b7ee59d8c694248a831b8cd529fc161
I20260811 09:14:21.563993 20293 ts_tablet_manager.cc:1939] T 4b7ee59d8c694248a831b8cd529fc161 P a0cd4f542eac492ab84dbb43327ef8ad: Deleting consensus metadata
I20260811 09:14:21.564147 20253 tablet_bootstrap.cc:492] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad: Bootstrap starting.
I20260811 09:14:21.565812 20253 tablet_bootstrap.cc:654] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:21.567335 19318 catalog_manager.cc:4841] TS a0cd4f542eac492ab84dbb43327ef8ad (127.17.16.65:42203): tablet 4b7ee59d8c694248a831b8cd529fc161 (table test-table [id=c20387bb71ac4c2ba6c5ac60cab0b7fb]) successfully deleted
I20260811 09:14:21.572750 19327 catalog_manager.cc:4841] TS 80f826ed499b4c23a3390b1f7b645f36 (127.17.16.67:43331): tablet 4b7ee59d8c694248a831b8cd529fc161 (table test-table [id=c20387bb71ac4c2ba6c5ac60cab0b7fb]) successfully deleted
I20260811 09:14:21.577977 20253 tablet_bootstrap.cc:492] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad: No bootstrap required, opened a new log
I20260811 09:14:21.578162 20253 ts_tablet_manager.cc:1397] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20260811 09:14:21.579041 20253 raft_consensus.cc:348] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.579283 20253 raft_consensus.cc:374] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:21.579382 20253 raft_consensus.cc:729] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0cd4f542eac492ab84dbb43327ef8ad, State: Initialized, Role: FOLLOWER
I20260811 09:14:21.579656 20253 consensus_queue.cc:260] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.580642 20255 tablet_bootstrap.cc:492] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158: Bootstrap starting.
I20260811 09:14:21.581207 20253 ts_tablet_manager.cc:1428] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 09:14:21.582267 20255 tablet_bootstrap.cc:654] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:21.591863 20254 tablet_bootstrap.cc:492] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36: Bootstrap starting.
I20260811 09:14:21.592808 20289 tablet_replica.cc:330] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: stopping tablet replica
I20260811 09:14:21.593423 20289 raft_consensus.cc:2229] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:21.593658 20254 tablet_bootstrap.cc:654] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:21.594071 20289 raft_consensus.cc:2258] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:21.595571 20255 tablet_bootstrap.cc:492] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158: No bootstrap required, opened a new log
I20260811 09:14:21.595757 20255 ts_tablet_manager.cc:1397] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260811 09:14:21.597561 20255 raft_consensus.cc:348] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.597848 20255 raft_consensus.cc:374] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:21.597975 20255 raft_consensus.cc:729] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcef4046fd5848f3a2f63470bf4a4158, State: Initialized, Role: FOLLOWER
I20260811 09:14:21.598243 20255 consensus_queue.cc:260] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.598843 20255 ts_tablet_manager.cc:1428] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 09:14:21.600553 20289 ts_tablet_manager.cc:1905] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:21.603099 20254 tablet_bootstrap.cc:492] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36: No bootstrap required, opened a new log
I20260811 09:14:21.603261 20254 ts_tablet_manager.cc:1397] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260811 09:14:21.604121 20254 raft_consensus.cc:348] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.604357 20254 raft_consensus.cc:374] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:21.604460 20254 raft_consensus.cc:729] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80f826ed499b4c23a3390b1f7b645f36, State: Initialized, Role: FOLLOWER
I20260811 09:14:21.604776 20254 consensus_queue.cc:260] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.605909 20254 ts_tablet_manager.cc:1428] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 09:14:21.606299 20266 raft_consensus.cc:482] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:21.606513 20266 raft_consensus.cc:504] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.608425 20266 leader_election.cc:288] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0cd4f542eac492ab84dbb43327ef8ad (127.17.16.65:42203), 80f826ed499b4c23a3390b1f7b645f36 (127.17.16.67:43331)
I20260811 09:14:21.611420 20289 ts_tablet_manager.cc:1918] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:14:21.611639 20289 log.cc:1196] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/4b7ee59d8c694248a831b8cd529fc161
I20260811 09:14:21.612205 20289 ts_tablet_manager.cc:1939] T 4b7ee59d8c694248a831b8cd529fc161 P dcef4046fd5848f3a2f63470bf4a4158: Deleting consensus metadata
I20260811 09:14:21.614281 19324 catalog_manager.cc:4841] TS dcef4046fd5848f3a2f63470bf4a4158 (127.17.16.66:44969): tablet 4b7ee59d8c694248a831b8cd529fc161 (table test-table [id=c20387bb71ac4c2ba6c5ac60cab0b7fb]) successfully deleted
I20260811 09:14:21.617477 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.609109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42203 (local address 127.17.16.66:35651)
0811 09:14:21.609736 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.609762 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.610031 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.610595 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.610600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.610624 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:21.610979 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.610990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.612380 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.612385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.614743 (+ 2358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.614758 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.616625 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.616632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.616807 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.616853 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.617014 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.617146 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":1165,"threads_started":1}
I20260811 09:14:21.617530 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.608721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:35651 (local address 127.17.16.65:42203)
0811 09:14:21.608925 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.608932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.609874 (+ 942us) server_negotiation.cc:407] Connection header received
0811 09:14:21.610209 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.610215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.610293 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.610360 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:21.611154 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.612221 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.614944 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.616412 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.616597 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.616673 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.617058 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.617197 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:14:21.618214 19643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9081a6162cc8470895568512f5a95d79" candidate_uuid: "dcef4046fd5848f3a2f63470bf4a4158" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" is_pre_election: true
I20260811 09:14:21.618480 19643 raft_consensus.cc:2454] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcef4046fd5848f3a2f63470bf4a4158 in term 0.
I20260811 09:14:21.618816 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.610101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:52113 (local address 127.17.16.67:43331)
0811 09:14:21.610260 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.610265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.611603 (+ 1338us) server_negotiation.cc:407] Connection header received
0811 09:14:21.611905 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.611910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.611976 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.612029 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:21.612990 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.614131 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.616383 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.617687 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.617824 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.617889 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.618308 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.618440 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:14:21.618867 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.609745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:43331 (local address 127.17.16.66:52113)
0811 09:14:21.611508 (+ 1763us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.611527 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.611755 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.612218 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.612223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.612247 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:21.612786 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.612799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.614317 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.614323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.616223 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.616237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.617976 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.617981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.618144 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.618173 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.618273 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.618393 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1636,"mutex_wait_us":4,"thread_start_us":1468,"threads_started":1}
I20260811 09:14:21.619021 19750 leader_election.cc:302] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0cd4f542eac492ab84dbb43327ef8ad, dcef4046fd5848f3a2f63470bf4a4158; no voters:
I20260811 09:14:21.619422 20266 raft_consensus.cc:2781] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:21.619550 20266 raft_consensus.cc:482] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:21.619642 20266 raft_consensus.cc:3037] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:21.619921 20129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9081a6162cc8470895568512f5a95d79" candidate_uuid: "dcef4046fd5848f3a2f63470bf4a4158" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80f826ed499b4c23a3390b1f7b645f36" is_pre_election: true
I20260811 09:14:21.620270 20129 raft_consensus.cc:2454] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcef4046fd5848f3a2f63470bf4a4158 in term 0.
I20260811 09:14:21.622702 20266 raft_consensus.cc:504] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.623360 20266 leader_election.cc:288] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [CANDIDATE]: Term 1 election: Requested vote from peers a0cd4f542eac492ab84dbb43327ef8ad (127.17.16.65:42203), 80f826ed499b4c23a3390b1f7b645f36 (127.17.16.67:43331)
I20260811 09:14:21.623835 20129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9081a6162cc8470895568512f5a95d79" candidate_uuid: "dcef4046fd5848f3a2f63470bf4a4158" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80f826ed499b4c23a3390b1f7b645f36"
I20260811 09:14:21.623804 19643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9081a6162cc8470895568512f5a95d79" candidate_uuid: "dcef4046fd5848f3a2f63470bf4a4158" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0cd4f542eac492ab84dbb43327ef8ad"
I20260811 09:14:21.623998 19643 raft_consensus.cc:3037] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:21.624017 20129 raft_consensus.cc:3037] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:21.626504 20129 raft_consensus.cc:2454] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcef4046fd5848f3a2f63470bf4a4158 in term 1.
I20260811 09:14:21.626885 19758 leader_election.cc:302] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80f826ed499b4c23a3390b1f7b645f36, dcef4046fd5848f3a2f63470bf4a4158; no voters:
I20260811 09:14:21.627151 20266 raft_consensus.cc:2781] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:21.627135 19643 raft_consensus.cc:2454] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcef4046fd5848f3a2f63470bf4a4158 in term 1.
I20260811 09:14:21.633188 20266 raft_consensus.cc:686] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 1 LEADER]: Becoming Leader. State: Replica: dcef4046fd5848f3a2f63470bf4a4158, State: Running, Role: LEADER
I20260811 09:14:21.633608 20266 consensus_queue.cc:237] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.637352 19353 catalog_manager.cc:5495] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 reported cstate change: term changed from 0 to 1, leader changed from <none> to dcef4046fd5848f3a2f63470bf4a4158 (127.17.16.66). New cstate: current_term: 1 leader_uuid: "dcef4046fd5848f3a2f63470bf4a4158" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } health_report { overall_health: UNKNOWN } } }
Aug 11 09:14:21 dist-test-slave-ncp9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439659, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:21.694397 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.676074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:43959 (local address 127.0.0.1:38374)
0811 09:14:21.676827 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.676847 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.677032 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.677415 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.677421 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.678331 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:21.678706 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.678721 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.680289 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.680295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.682168 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.682184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.683419 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.683426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.683631 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.684409 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:21.684431 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:21.687569 (+ 3138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:21.691374 (+ 3805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:21.691389 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:21.691397 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:21.691963 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:21.692353 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:21.692358 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:21.692362 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:21.692549 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:21.693245 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:21.693254 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:21.693453 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.693641 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.693944 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":418,"threads_started":1}
I20260811 09:14:21.694449 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.675784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.17.16.126:43959)
0811 09:14:21.675998 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.676004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.676954 (+ 950us) server_negotiation.cc:407] Connection header received
0811 09:14:21.677154 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.677159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.677249 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.677351 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:21.678878 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.680116 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.682352 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.683223 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.683351 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.687774 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:21.687781 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:21.687786 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:21.687826 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:21.691221 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:21.692096 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:21.692102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:21.692105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:21.692249 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:21.692680 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:21.692685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:21.692688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:21.693096 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:21.693234 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.693773 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.694115 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":51}
I20260811 09:14:21.700977 19353 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:38374:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 09:14:21.702567 19353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:21.709645 19602 tablet_service.cc:1467] Processing CreateTablet for tablet b7da8fe7f0b74e41b1c67d176aa7f11d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e2b0ffff474549eabde7a27827b80016]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:14:21.709873 19836 tablet_service.cc:1467] Processing CreateTablet for tablet b7da8fe7f0b74e41b1c67d176aa7f11d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e2b0ffff474549eabde7a27827b80016]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:14:21.710278 19602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7da8fe7f0b74e41b1c67d176aa7f11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:21.710456 19836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7da8fe7f0b74e41b1c67d176aa7f11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:21.714212 20089 tablet_service.cc:1467] Processing CreateTablet for tablet b7da8fe7f0b74e41b1c67d176aa7f11d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e2b0ffff474549eabde7a27827b80016]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:14:21.714854 20089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7da8fe7f0b74e41b1c67d176aa7f11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:21.716708 20255 tablet_bootstrap.cc:492] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: Bootstrap starting.
I20260811 09:14:21.717429 20253 tablet_bootstrap.cc:492] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: Bootstrap starting.
I20260811 09:14:21.718623 20255 tablet_bootstrap.cc:654] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:21.719216 20253 tablet_bootstrap.cc:654] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:21.723387 20254 tablet_bootstrap.cc:492] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: Bootstrap starting.
I20260811 09:14:21.725178 20254 tablet_bootstrap.cc:654] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:21.725451 20255 tablet_bootstrap.cc:492] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: No bootstrap required, opened a new log
I20260811 09:14:21.725623 20255 ts_tablet_manager.cc:1397] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260811 09:14:21.726090 20253 tablet_bootstrap.cc:492] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: No bootstrap required, opened a new log
I20260811 09:14:21.726316 20253 ts_tablet_manager.cc:1397] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.003s
I20260811 09:14:21.726393 20255 raft_consensus.cc:348] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.726650 20255 raft_consensus.cc:374] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:21.726737 20255 raft_consensus.cc:729] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcef4046fd5848f3a2f63470bf4a4158, State: Initialized, Role: FOLLOWER
I20260811 09:14:21.727022 20255 consensus_queue.cc:260] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.727334 20253 raft_consensus.cc:348] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.727626 20253 raft_consensus.cc:374] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:21.727506 20266 tablet_replica.cc:439] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } } }
I20260811 09:14:21.727793 20266 tablet_replica.cc:442] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:21.727809 20253 raft_consensus.cc:729] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0cd4f542eac492ab84dbb43327ef8ad, State: Initialized, Role: FOLLOWER
I20260811 09:14:21.728205 20253 consensus_queue.cc:260] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.728777 20264 tablet_replica.cc:439] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } } }
I20260811 09:14:21.729300 20264 tablet_replica.cc:442] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:21.730334 20253 ts_tablet_manager.cc:1428] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 09:14:21.731189 20255 ts_tablet_manager.cc:1428] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 09:14:21.732921 20254 tablet_bootstrap.cc:492] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: No bootstrap required, opened a new log
I20260811 09:14:21.733356 20254 ts_tablet_manager.cc:1397] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260811 09:14:21.734194 20254 raft_consensus.cc:348] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.734447 20254 raft_consensus.cc:374] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:21.734627 20254 raft_consensus.cc:729] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80f826ed499b4c23a3390b1f7b645f36, State: Initialized, Role: FOLLOWER
I20260811 09:14:21.734929 20254 consensus_queue.cc:260] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.735437 20254 ts_tablet_manager.cc:1428] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 09:14:21.735369 20262 tablet_replica.cc:439] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } } }
I20260811 09:14:21.735843 20262 tablet_replica.cc:442] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:21.849735 20264 raft_consensus.cc:482] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:21.849990 20264 raft_consensus.cc:504] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.851780 20264 leader_election.cc:288] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dcef4046fd5848f3a2f63470bf4a4158 (127.17.16.66:44969), 80f826ed499b4c23a3390b1f7b645f36 (127.17.16.67:43331)
I20260811 09:14:21.861317 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.852603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:44937 (local address 127.17.16.67:43331)
0811 09:14:21.852805 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.852811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.854199 (+ 1388us) server_negotiation.cc:407] Connection header received
0811 09:14:21.854532 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.854537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.854613 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.854703 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:21.855428 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.856465 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.858667 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.860174 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.860334 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.860412 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.860857 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.861022 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:14:21.861358 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.852255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:39559 (local address 127.17.16.66:44969)
0811 09:14:21.852436 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.852442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.853753 (+ 1311us) server_negotiation.cc:407] Connection header received
0811 09:14:21.853923 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.853927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.853988 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.854060 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:21.854981 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.856090 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.858887 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.860371 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.860515 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.860583 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.860998 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.861120 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:14:21.861382 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.852611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:43331 (local address 127.17.16.65:44937)
0811 09:14:21.854084 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.854104 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.854339 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.854760 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.854768 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.854799 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:21.855236 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.855251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.856584 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.856588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.858487 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.858500 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.860399 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.860406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.860592 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.860700 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.860796 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.860966 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"mutex_wait_us":53,"spinlock_wait_cycles":16896,"thread_start_us":581,"threads_started":1}
I20260811 09:14:21.862107 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.852256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:44969 (local address 127.17.16.65:39559)
0811 09:14:21.853628 (+ 1372us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.853656 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.853774 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.854106 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.854110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.854139 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:21.854808 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.854823 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.856245 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.856251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.858729 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.858746 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.860604 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.860610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.860795 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.860828 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.861584 (+ 756us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.861724 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"mutex_wait_us":68,"spinlock_wait_cycles":23808,"thread_start_us":1085,"threads_started":1}
I20260811 09:14:21.862269 20129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7da8fe7f0b74e41b1c67d176aa7f11d" candidate_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80f826ed499b4c23a3390b1f7b645f36" is_pre_election: true
I20260811 09:14:21.862581 20129 raft_consensus.cc:2454] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0cd4f542eac492ab84dbb43327ef8ad in term 0.
I20260811 09:14:21.862825 19872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7da8fe7f0b74e41b1c67d176aa7f11d" candidate_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcef4046fd5848f3a2f63470bf4a4158" is_pre_election: true
I20260811 09:14:21.863029 19509 leader_election.cc:302] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80f826ed499b4c23a3390b1f7b645f36, a0cd4f542eac492ab84dbb43327ef8ad; no voters:
I20260811 09:14:21.863085 19872 raft_consensus.cc:2454] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0cd4f542eac492ab84dbb43327ef8ad in term 0.
I20260811 09:14:21.863360 20264 raft_consensus.cc:2781] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:21.863481 20264 raft_consensus.cc:482] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:21.863587 20264 raft_consensus.cc:3037] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:21.865820 20264 raft_consensus.cc:504] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.866408 20264 leader_election.cc:288] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [CANDIDATE]: Term 1 election: Requested vote from peers dcef4046fd5848f3a2f63470bf4a4158 (127.17.16.66:44969), 80f826ed499b4c23a3390b1f7b645f36 (127.17.16.67:43331)
I20260811 09:14:21.866894 19872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7da8fe7f0b74e41b1c67d176aa7f11d" candidate_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcef4046fd5848f3a2f63470bf4a4158"
I20260811 09:14:21.866938 20129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7da8fe7f0b74e41b1c67d176aa7f11d" candidate_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80f826ed499b4c23a3390b1f7b645f36"
I20260811 09:14:21.867121 19872 raft_consensus.cc:3037] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:21.867180 20129 raft_consensus.cc:3037] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:21.870216 19872 raft_consensus.cc:2454] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0cd4f542eac492ab84dbb43327ef8ad in term 1.
I20260811 09:14:21.870323 20129 raft_consensus.cc:2454] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0cd4f542eac492ab84dbb43327ef8ad in term 1.
I20260811 09:14:21.870668 19502 leader_election.cc:302] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0cd4f542eac492ab84dbb43327ef8ad, dcef4046fd5848f3a2f63470bf4a4158; no voters:
I20260811 09:14:21.870961 20264 raft_consensus.cc:2781] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:21.871451 20264 raft_consensus.cc:686] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [term 1 LEADER]: Becoming Leader. State: Replica: a0cd4f542eac492ab84dbb43327ef8ad, State: Running, Role: LEADER
I20260811 09:14:21.871879 20264 consensus_queue.cc:237] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } }
I20260811 09:14:21.874338 20327 tablet_replica.cc:439] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: TxnStatusTablet state changed. Reason: New leader a0cd4f542eac492ab84dbb43327ef8ad. Latest consensus state: current_term: 1 leader_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } } }
I20260811 09:14:21.874701 20327 tablet_replica.cc:442] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:14:21.875785 19353 catalog_manager.cc:5495] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad reported cstate change: term changed from 0 to 1, leader changed from <none> to a0cd4f542eac492ab84dbb43327ef8ad (127.17.16.65). New cstate: current_term: 1 leader_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } health_report { overall_health: UNKNOWN } } }
I20260811 09:14:21.877460 20329 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 09:14:21.877789 20329 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 09:14:21.928117 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.922126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42203 (local address 127.0.0.1:55240)
0811 09:14:21.922488 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.922509 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.922616 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.923366 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.923371 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.923383 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:14:21.923553 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.923562 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.924662 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.924668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.925719 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.925732 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.926453 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.926458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.926607 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.926672 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:14:21.927482 (+ 810us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:14:21.927505 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.927584 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.927745 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260811 09:14:21.928158 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.922844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55240 (local address 127.17.16.65:42203)
0811 09:14:21.923007 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.923012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.923026 (+ 14us) server_negotiation.cc:407] Connection header received
0811 09:14:21.923103 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.923107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.923189 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.923317 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:21.923691 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.924527 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.925866 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.926308 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.926417 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.926842 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:21.927373 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:21.927509 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.927624 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.927704 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:14:21.935456 19872 raft_consensus.cc:1261] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158 [term 1 FOLLOWER]: Refusing update from remote peer a0cd4f542eac492ab84dbb43327ef8ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:21.935621 20129 raft_consensus.cc:1261] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36 [term 1 FOLLOWER]: Refusing update from remote peer a0cd4f542eac492ab84dbb43327ef8ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:21.936293 20264 consensus_queue.cc:1035] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:21.937016 20327 consensus_queue.cc:1035] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:21.941130 20266 tablet_replica.cc:439] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: TxnStatusTablet state changed. Reason: New leader a0cd4f542eac492ab84dbb43327ef8ad. Latest consensus state: current_term: 1 leader_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } } }
I20260811 09:14:21.941437 20266 tablet_replica.cc:442] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:21.941856 20327 tablet_replica.cc:439] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } } }
I20260811 09:14:21.942121 20327 tablet_replica.cc:442] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:14:21.945246 20275 mvcc.cc:205] Tried to move back new op lower bound from 7317256855279296512 to 7317256855030775808. Current Snapshot: MvccSnapshot[applied={T|T < 7317256855279296512 or (T in {7317256855279296512})}]
I20260811 09:14:21.947952 20276 mvcc.cc:205] Tried to move back new op lower bound from 7317256855279296512 to 7317256855030775808. Current Snapshot: MvccSnapshot[applied={T|T < 7317256855279296512}]
I20260811 09:14:21.948387 20266 tablet_replica.cc:439] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } } }
I20260811 09:14:21.948683 20266 tablet_replica.cc:442] T b7da8fe7f0b74e41b1c67d176aa7f11d P dcef4046fd5848f3a2f63470bf4a4158: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:21.960246 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.951419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40430 (local address 127.17.16.66:44969)
0811 09:14:21.951591 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.951597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.951616 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:14:21.951732 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.951738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.951838 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.951986 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:21.952576 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.953811 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.956204 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.956998 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.957136 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.957931 (+ 795us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:21.958956 (+ 1025us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:21.959114 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.959830 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.959930 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:14:21.963829 20264 tablet_replica.cc:439] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } } }
I20260811 09:14:21.964165 20264 tablet_replica.cc:442] T b7da8fe7f0b74e41b1c67d176aa7f11d P a0cd4f542eac492ab84dbb43327ef8ad: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:14:21.966712 20262 tablet_replica.cc:439] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: TxnStatusTablet state changed. Reason: New leader a0cd4f542eac492ab84dbb43327ef8ad. Latest consensus state: current_term: 1 leader_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } } }
I20260811 09:14:21.967262 20262 tablet_replica.cc:442] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:21.970846 20262 tablet_replica.cc:439] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 } } peers { permanent_uuid: "dcef4046fd5848f3a2f63470bf4a4158" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44969 } } peers { permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 } } }
I20260811 09:14:21.971127 20262 tablet_replica.cc:442] T b7da8fe7f0b74e41b1c67d176aa7f11d P 80f826ed499b4c23a3390b1f7b645f36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:21.994980 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.980497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:43005 (local address 127.17.16.65:42203)
0811 09:14:21.980750 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:21.980758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:21.987955 (+ 7197us) server_negotiation.cc:407] Connection header received
0811 09:14:21.988249 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:21.988256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:21.988351 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:21.988460 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:21.989175 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.990330 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.992444 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.993274 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.993399 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.994021 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:21.994179 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:21.994277 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:21.994425 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 09:14:21.994510 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 09:14:21.995016 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:21.980379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42203 (local address 127.17.16.66:43005)
0811 09:14:21.987795 (+ 7416us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:21.987821 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:21.988079 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:21.988508 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:21.988514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:21.988529 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:14:21.988975 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.989013 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.990512 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.990520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.992266 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:21.992285 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:21.993507 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:21.993514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:21.993719 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:21.993811 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:14:21.994292 (+ 481us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:14:21.994307 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 09:14:21.994403 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 09:14:21.994566 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7151,"thread_start_us":6974,"threads_started":1}
I20260811 09:14:22.017469 20129 raft_consensus.cc:1261] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36 [term 1 FOLLOWER]: Refusing update from remote peer dcef4046fd5848f3a2f63470bf4a4158: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:22.017532 19643 raft_consensus.cc:1261] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad [term 1 FOLLOWER]: Refusing update from remote peer dcef4046fd5848f3a2f63470bf4a4158: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:22.018989 20266 consensus_queue.cc:1035] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80f826ed499b4c23a3390b1f7b645f36" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:22.019975 20304 consensus_queue.cc:1035] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0cd4f542eac492ab84dbb43327ef8ad" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:22.134320 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:22.121230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:52043 (local address 127.17.16.66:44969)
0811 09:14:22.121564 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:22.121573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:22.123627 (+ 2054us) server_negotiation.cc:407] Connection header received
0811 09:14:22.124730 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:22.124744 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:22.124853 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:22.125026 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:22.127008 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.128215 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.130636 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.131618 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.131818 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:22.132585 (+ 767us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:22.132831 (+ 246us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:22.133041 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:22.133328 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 09:14:22.133478 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 09:14:22.135169 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:22.120271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:44969 (local address 127.17.16.65:52043)
0811 09:14:22.123457 (+ 3186us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:22.123504 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:22.124402 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:22.125059 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:22.125064 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:22.125106 (+ 42us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:14:22.126625 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:22.126654 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.128539 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.128551 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:22.130439 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:22.130470 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.131926 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.131944 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:22.132149 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:22.132267 (+ 118us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:14:22.133064 (+ 797us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:14:22.133096 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:14:22.133297 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 09:14:22.133574 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2670,"thread_start_us":2367,"threads_started":1}
I20260811 09:14:22.236372 19353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:22.236866 19353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:22.244988 19353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 267f2c0cbc2b481aafb3b3e91c841cf3: Sending DeleteTablet for 3 replicas of tablet 9081a6162cc8470895568512f5a95d79
I20260811 09:14:22.246644 19602 tablet_service.cc:1514] Processing DeleteTablet for tablet 9081a6162cc8470895568512f5a95d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:22 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:55234
I20260811 09:14:22.249455 19836 tablet_service.cc:1514] Processing DeleteTablet for tablet 9081a6162cc8470895568512f5a95d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:22 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:40424
I20260811 09:14:22.250995 20089 tablet_service.cc:1514] Processing DeleteTablet for tablet 9081a6162cc8470895568512f5a95d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:22 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:48730
I20260811 09:14:22.253605 20359 tablet_replica.cc:330] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad: stopping tablet replica
I20260811 09:14:22.254009 20359 raft_consensus.cc:2229] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:22.254742 20359 raft_consensus.cc:2258] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:22.256968 20359 ts_tablet_manager.cc:1905] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:22.265653 20363 tablet_replica.cc:330] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36: stopping tablet replica
I20260811 09:14:22.266067 20363 raft_consensus.cc:2229] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:22.266510 20363 raft_consensus.cc:2258] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:22.272871 20363 ts_tablet_manager.cc:1905] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:22.278947 20359 ts_tablet_manager.cc:1918] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:14:22.279160 20361 tablet_replica.cc:330] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158: stopping tablet replica
I20260811 09:14:22.279193 20359 log.cc:1196] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/9081a6162cc8470895568512f5a95d79
I20260811 09:14:22.279592 20361 raft_consensus.cc:2229] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:14:22.280078 20359 ts_tablet_manager.cc:1939] T 9081a6162cc8470895568512f5a95d79 P a0cd4f542eac492ab84dbb43327ef8ad: Deleting consensus metadata
I20260811 09:14:22.280671 20361 raft_consensus.cc:2258] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:22.282929 19318 catalog_manager.cc:4841] TS a0cd4f542eac492ab84dbb43327ef8ad (127.17.16.65:42203): tablet 9081a6162cc8470895568512f5a95d79 (table test-table [id=80bae4aa8446414a8cdbce0ec486932b]) successfully deleted
I20260811 09:14:22.285018 20361 ts_tablet_manager.cc:1905] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:22.290815 20363 ts_tablet_manager.cc:1918] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:14:22.291169 20363 log.cc:1196] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/9081a6162cc8470895568512f5a95d79
I20260811 09:14:22.292207 20363 ts_tablet_manager.cc:1939] T 9081a6162cc8470895568512f5a95d79 P 80f826ed499b4c23a3390b1f7b645f36: Deleting consensus metadata
I20260811 09:14:22.294548 19327 catalog_manager.cc:4841] TS 80f826ed499b4c23a3390b1f7b645f36 (127.17.16.67:43331): tablet 9081a6162cc8470895568512f5a95d79 (table test-table [id=80bae4aa8446414a8cdbce0ec486932b]) successfully deleted
Aug 11 09:14:22 dist-test-slave-ncp9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439661, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:14:22.303534 20361 ts_tablet_manager.cc:1918] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:14:22.304162 20361 log.cc:1196] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/9081a6162cc8470895568512f5a95d79
I20260811 09:14:22.305202 20361 ts_tablet_manager.cc:1939] T 9081a6162cc8470895568512f5a95d79 P dcef4046fd5848f3a2f63470bf4a4158: Deleting consensus metadata
I20260811 09:14:22.307473 19324 catalog_manager.cc:4841] TS dcef4046fd5848f3a2f63470bf4a4158 (127.17.16.66:44969): tablet 9081a6162cc8470895568512f5a95d79 (table test-table [id=80bae4aa8446414a8cdbce0ec486932b]) successfully deleted
I20260811 09:14:22.314079 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:22.273989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.17.16.65:42203)
0811 09:14:22.274501 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:22.274535 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:22.279796 (+ 5261us) server_negotiation.cc:407] Connection header received
0811 09:14:22.280135 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:22.280155 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:22.280287 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:22.280424 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:22.284586 (+ 4162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.286361 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.289967 (+ 3606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.290987 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.291182 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:22.300994 (+ 9812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:22.301026 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:22.301068 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:22.301144 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:22.308567 (+ 7423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:22.310123 (+ 1556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:22.310147 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:22.310150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:22.310387 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:22.311152 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:22.311160 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:22.311163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:22.311780 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:22.312318 (+ 538us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:22.312882 (+ 564us) server_negotiation.cc:299] Negotiation successful
0811 09:14:22.313342 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
W20260811 09:14:22.315393 19715 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55242
I20260811 09:14:22.355000 20371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:22.322719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38382 (local address 127.17.16.126:43959)
0811 09:14:22.331058 (+ 8339us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:22.331064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:22.331092 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:14:22.331188 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:22.331193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:22.331296 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:22.331488 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:22.333998 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.335423 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.337780 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.338476 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.338701 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:22.342788 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:22.342794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:22.342798 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:22.342871 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:22.348543 (+ 5672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:22.350124 (+ 1581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:22.350129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:22.350167 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:22.350508 (+ 341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:22.351150 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:22.351160 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:22.351163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:22.351746 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:22.352302 (+ 556us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:22.353211 (+ 909us) server_negotiation.cc:299] Negotiation successful
0811 09:14:22.353933 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8150,"thread_start_us":7913,"threads_started":1}
W20260811 09:14:22.356251 19353 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38382
I20260811 09:14:22.361501 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19470
I20260811 09:14:22.391800 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:22.367647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.17.16.65:42203)
0811 09:14:22.368404 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:22.368424 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:22.369801 (+ 1377us) server_negotiation.cc:407] Connection header received
0811 09:14:22.370160 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:22.370165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:22.370277 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:22.370410 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:22.372344 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.373676 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.375652 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.376463 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.376671 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:22.380878 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:22.380883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:22.380940 (+ 57us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:22.381011 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:22.385917 (+ 4906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:22.386991 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:22.387000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:22.387004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:22.387380 (+ 376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:22.387964 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:22.387970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:22.388003 (+ 33us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:22.388617 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:22.389974 (+ 1357us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:22.390453 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 09:14:22.390938 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490}
W20260811 09:14:22.392769 19715 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55244
W20260811 09:14:22.393899 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19470: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:14:22.394244 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 19470
I20260811 09:14:22.450558 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19730
Aug 11 09:14:22 dist-test-slave-ncp9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439661, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:14:22.480780 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:22.451603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.17.16.66:44969)
0811 09:14:22.451914 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:22.451933 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:22.452031 (+ 98us) server_negotiation.cc:407] Connection header received
0811 09:14:22.452383 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:22.452387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:22.452505 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:22.452673 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:22.455534 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.456798 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.460161 (+ 3363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.460876 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.461007 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:22.470552 (+ 9545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:22.470581 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:22.470610 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:22.470684 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:22.475883 (+ 5199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:22.477162 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:22.477180 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:22.477186 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:22.477433 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:22.478476 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:22.478481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:22.478483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:22.479075 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:22.479416 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:22.479940 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 09:14:22.480284 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
W20260811 09:14:22.481941 19968 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40434
W20260811 09:14:22.483017 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19730: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:14:22.483309 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 19730
I20260811 09:14:22.515190 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19983
Aug 11 09:14:22 dist-test-slave-ncp9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439661, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:14:22.538087 20374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:22.516137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48742 (local address 127.17.16.67:43331)
0811 09:14:22.516926 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:22.516940 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:22.516965 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:14:22.517046 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:22.517054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:22.517141 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:22.517291 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:22.519068 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.520119 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.522028 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:22.522853 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:22.523028 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:22.528944 (+ 5916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:22.528968 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:22.528994 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:22.529066 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:22.533991 (+ 4925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:22.534962 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:22.534978 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:22.534980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:22.535171 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:22.535801 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:22.535805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:22.535807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:22.536360 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:22.536800 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:22.537254 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 09:14:22.537581 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":373,"threads_started":1}
W20260811 09:14:22.539005 20204 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48742
W20260811 09:14:22.539856 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19983: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:14:22.540119 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 19983
I20260811 09:14:22.582870 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19289
I20260811 09:14:22.584442 19425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:22.761114 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 19289
2026-08-11T09:14:22Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5106 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:14:23 dist-test-slave-ncp9 krb5kdc[20380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:14:23 dist-test-slave-ncp9 krb5kdc[20380](info): set up 2 sockets
Aug 11 09:14:23 dist-test-slave-ncp9 krb5kdc[20380](info): commencing operation
krb5kdc: starting...
W20260811 09:14:24.400167 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.497s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:14:24 dist-test-slave-ncp9 krb5kdc[20380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:14:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:14:24.609552 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:39149
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:39149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:24.935578 20403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:24.935894 20403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:24.936000 20403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:24.944269 20403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:24.944386 20403 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 09:14:24.944432 20403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:24.944500 20403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:24.944550 20403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:24.944612 20403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:24.954782 20403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:39149
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:39149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:24.956084 20403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:24.957444 20403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:24.968849 20410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:24.968858 20413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:24.968767 20416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:24.969969 20403 server_base.cc:1034] running on GCE node
I20260811 09:14:24.970655 20403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:24.971920 20403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:24.973130 20403 hybrid_clock.cc:648] HybridClock initialized: now 1786439664973069 us; error 32 us; skew 500 ppm
I20260811 09:14:24.973439 20403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:24 dist-test-slave-ncp9 krb5kdc[20380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439664, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:24.976372 20403 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:14:24.977782 20403 webserver.cc:460] Webserver started at http://127.17.16.126:45461/ using document root <none> and password file <none>
I20260811 09:14:24.978444 20403 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:24.978554 20403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:24.978844 20403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:24.981766 20403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "5ca67e87a37f4d008389631f1af75a62"
format_stamp: "Formatted at 2026-08-11 09:14:24 on dist-test-slave-ncp9"
server_key: "189b9efbb4ad48d9a63b3c649544533a"
server_key_iv: "a6e997e79a41f6918692ff3a0e9e8ce5"
server_key_version: "encryptionkey@0"
I20260811 09:14:24.982630 20403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "5ca67e87a37f4d008389631f1af75a62"
format_stamp: "Formatted at 2026-08-11 09:14:24 on dist-test-slave-ncp9"
server_key: "189b9efbb4ad48d9a63b3c649544533a"
server_key_iv: "a6e997e79a41f6918692ff3a0e9e8ce5"
server_key_version: "encryptionkey@0"
I20260811 09:14:24.987762 20403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:14:24.991196 20425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:24.992449 20403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:14:24.992622 20403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "5ca67e87a37f4d008389631f1af75a62"
format_stamp: "Formatted at 2026-08-11 09:14:24 on dist-test-slave-ncp9"
server_key: "189b9efbb4ad48d9a63b3c649544533a"
server_key_iv: "a6e997e79a41f6918692ff3a0e9e8ce5"
server_key_version: "encryptionkey@0"
I20260811 09:14:24.992823 20403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:25.023950 20403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:25.028255 20403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:25.028563 20403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:25.050427 20403 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:39149
I20260811 09:14:25.050465 20538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:39149 every 8 connection(s)
I20260811 09:14:25.051834 20403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:25.055455 20539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:25.056674 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 20403
I20260811 09:14:25.056872 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:25.057607 17473 external_mini_cluster.cc:1428] Setting key 32b1b4d19e8762f38c11164ebf6e7910
I20260811 09:14:25.066721 20539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62: Bootstrap starting.
I20260811 09:14:25.069865 20539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:25.070854 20539 log.cc:824] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:14:25 dist-test-slave-ncp9 krb5kdc[20380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:25.073967 20539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62: No bootstrap required, opened a new log
I20260811 09:14:25.081965 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:25.059477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.17.16.126:39149)
0811 09:14:25.060306 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:25.060324 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:25.060694 (+ 370us) server_negotiation.cc:407] Connection header received
0811 09:14:25.062680 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:25.062702 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:25.063067 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:25.063420 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:25.065045 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:25.066402 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:25.067788 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:25.068546 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:25.068733 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:25.073083 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:25.073096 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:25.073117 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:25.073174 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:25.077268 (+ 4094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:25.077936 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:25.077946 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:25.077955 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:25.078189 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:25.078512 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:25.078523 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:25.078527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:25.079280 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:25.079657 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:25.079985 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 09:14:25.080474 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":354,"threads_started":1}
I20260811 09:14:25.085080 20539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca67e87a37f4d008389631f1af75a62" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39149 } }
I20260811 09:14:25.085445 20539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:25.085536 20539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ca67e87a37f4d008389631f1af75a62, State: Initialized, Role: FOLLOWER
I20260811 09:14:25.086062 20539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca67e87a37f4d008389631f1af75a62" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39149 } }
I20260811 09:14:25.086292 20539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:25.086407 20539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:25.086571 20539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:25.089134 20539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca67e87a37f4d008389631f1af75a62" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39149 } }
I20260811 09:14:25.089659 20539 leader_election.cc:302] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ca67e87a37f4d008389631f1af75a62; no voters:
I20260811 09:14:25.090235 20539 leader_election.cc:288] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:25.090340 20548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:25.091053 20548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [term 1 LEADER]: Becoming Leader. State: Replica: 5ca67e87a37f4d008389631f1af75a62, State: Running, Role: LEADER
I20260811 09:14:25.091585 20548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca67e87a37f4d008389631f1af75a62" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39149 } }
I20260811 09:14:25.092432 20539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:25.094229 20550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ca67e87a37f4d008389631f1af75a62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca67e87a37f4d008389631f1af75a62" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39149 } } }
I20260811 09:14:25.094585 20550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:25.095198 20551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ca67e87a37f4d008389631f1af75a62. Latest consensus state: current_term: 1 leader_uuid: "5ca67e87a37f4d008389631f1af75a62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca67e87a37f4d008389631f1af75a62" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39149 } } }
I20260811 09:14:25.095611 20551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:25.099381 20558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:25.103842 20558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:25.111707 20558 catalog_manager.cc:1347] Generated new cluster ID: c2944a1e30fa412eb468b9bf52b31612
I20260811 09:14:25.111824 20558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:25.122000 20558 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:25.123924 20558 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:25.134402 20558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62: Generated new TSK 0
I20260811 09:14:25.135428 20558 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:14:25.184837 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:39149
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:25.512809 20584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:25.513129 20584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:25.513283 20584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:25.520392 20584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:25.520503 20584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:25.520632 20584 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 09:14:25.520706 20584 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:14:25.520809 20584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:14:25.530449 20584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.16.126:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:25.531726 20584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:25.533037 20584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:25.549229 20595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:25.549145 20601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:25.549175 20598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:25.549970 20584 server_base.cc:1034] running on GCE node
I20260811 09:14:25.550575 20584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:25.551506 20584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:25.552699 20584 hybrid_clock.cc:648] HybridClock initialized: now 1786439665552663 us; error 46 us; skew 500 ppm
I20260811 09:14:25.553031 20584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:25 dist-test-slave-ncp9 krb5kdc[20380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439665, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:25.556169 20584 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:14:25.557575 20584 webserver.cc:460] Webserver started at http://127.17.16.65:35025/ using document root <none> and password file <none>
I20260811 09:14:25.558205 20584 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:25.558305 20584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:25.558564 20584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:25.561431 20584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "4dee37677845496fb27c1256f148d60d"
format_stamp: "Formatted at 2026-08-11 09:14:25 on dist-test-slave-ncp9"
server_key: "8fdbd6c1ae0245c6cab78e50af121720"
server_key_iv: "6c3e8e253f94377d467d1ea188e1ef78"
server_key_version: "encryptionkey@0"
I20260811 09:14:25.562139 20584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "4dee37677845496fb27c1256f148d60d"
format_stamp: "Formatted at 2026-08-11 09:14:25 on dist-test-slave-ncp9"
server_key: "8fdbd6c1ae0245c6cab78e50af121720"
server_key_iv: "6c3e8e253f94377d467d1ea188e1ef78"
server_key_version: "encryptionkey@0"
I20260811 09:14:25.567378 20584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:14:25.570713 20618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:25.571933 20584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:14:25.572063 20584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "4dee37677845496fb27c1256f148d60d"
format_stamp: "Formatted at 2026-08-11 09:14:25 on dist-test-slave-ncp9"
server_key: "8fdbd6c1ae0245c6cab78e50af121720"
server_key_iv: "6c3e8e253f94377d467d1ea188e1ef78"
server_key_version: "encryptionkey@0"
I20260811 09:14:25.572240 20584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:25.602670 20584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:25.606803 20584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:25.607079 20584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:25.609776 20584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:25.609894 20584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:25.610026 20584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:25.610112 20584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:14:25 dist-test-slave-ncp9 krb5kdc[20380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439665, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:25.633463 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:25.610162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:55559 (local address 127.17.16.126:39149)
0811 09:14:25.610627 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:25.610634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:25.611773 (+ 1139us) server_negotiation.cc:407] Connection header received
0811 09:14:25.613780 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:25.613786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:25.613867 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:25.613960 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:25.616032 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:25.617065 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:25.619942 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:25.620474 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:25.620615 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:25.625963 (+ 5348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:25.625969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:25.625973 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:25.626008 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:25.629388 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:25.630619 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:25.630625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:25.630629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:25.630794 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:25.631314 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:25.631319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:25.631321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:25.631644 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:25.631801 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:25.632972 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0811 09:14:25.633197 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":235,"threads_started":1}
I20260811 09:14:25.635138 20632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:25.610850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39149 (local address 127.17.16.65:55559)
0811 09:14:25.611630 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:25.611670 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:25.613514 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:25.614204 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:25.614214 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:25.614959 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:25.615840 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:25.615863 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:25.617257 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:25.617263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:25.619786 (+ 2523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:25.619803 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:25.620731 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:25.620739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:25.621090 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:25.621963 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:25.621995 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:25.625645 (+ 3650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:25.629587 (+ 3942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:25.629597 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:25.629621 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:25.630430 (+ 809us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:25.630931 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:25.630940 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:25.630943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:25.631187 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:25.631788 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:25.631797 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:25.632403 (+ 606us) client_negotiation.cc:769] Sending connection context
0811 09:14:25.632839 (+ 436us) client_negotiation.cc:240] Negotiation successful
0811 09:14:25.633531 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":325,"threads_started":1}
I20260811 09:14:25.655282 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:25.641246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39149 (local address 127.17.16.65:41223)
0811 09:14:25.641851 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:25.641870 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:25.641983 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:25.642309 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:25.642312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:25.642859 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:25.643143 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:25.643152 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:25.644605 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:25.644609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:25.645962 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:25.645979 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:25.646959 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:25.646966 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:25.647136 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:25.647768 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:25.647791 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:25.648812 (+ 1021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:25.652445 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:25.652449 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:25.652452 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:25.652834 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:25.653251 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:25.653257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:25.653261 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:25.653462 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:25.654060 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:25.654065 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:25.654280 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 09:14:25.654427 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 09:14:25.654769 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":307,"threads_started":1}
I20260811 09:14:25.656278 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:25.641074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:41223 (local address 127.17.16.126:39149)
0811 09:14:25.641270 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:25.641276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:25.641953 (+ 677us) server_negotiation.cc:407] Connection header received
0811 09:14:25.642083 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:25.642089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:25.642180 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:25.642299 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:25.643307 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:25.644475 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:25.646235 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:25.646771 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:25.646882 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:25.649032 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:25.649039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:25.649044 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:25.649082 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:25.652319 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:25.653002 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:25.653006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:25.653009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:25.653136 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:25.653591 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:25.653596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:25.653600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:25.653925 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:25.654063 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:25.655724 (+ 1661us) server_negotiation.cc:299] Negotiation successful
0811 09:14:25.655951 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:14:25.658394 20584 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:45449
I20260811 09:14:25.658396 20822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:45449 every 8 connection(s)
I20260811 09:14:25.660017 20584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:14:25.662073 20826 heartbeater.cc:347] Connected to a master server at 127.17.16.126:39149
I20260811 09:14:25.662333 20826 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:25.663012 20826 heartbeater.cc:511] Master 127.17.16.126:39149 requested a full tablet report, sending...
I20260811 09:14:25.664848 20453 ts_manager.cc:194] Registered new tserver with Master: 4dee37677845496fb27c1256f148d60d (127.17.16.65:45449)
I20260811 09:14:25.666338 20453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:55559
I20260811 09:14:25.666869 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 20584
I20260811 09:14:25.667070 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:14:25.667786 17473 external_mini_cluster.cc:1428] Setting key a5f1fceb84286fece09da47a85383d0a
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:14:25.712287 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:39149
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:26.035760 20835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:26.036067 20835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:26.036226 20835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:26.043359 20835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:26.043483 20835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:26.043619 20835 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 09:14:26.043687 20835 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:14:26.043789 20835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:14:26.053269 20835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.16.126:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:26.054479 20835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:26.055716 20835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:26.068535 20848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:26.068642 20843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:26.068828 20841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:26.069666 20835 server_base.cc:1034] running on GCE node
I20260811 09:14:26.070297 20835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:26.071174 20835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:26.072363 20835 hybrid_clock.cc:648] HybridClock initialized: now 1786439666072319 us; error 33 us; skew 500 ppm
I20260811 09:14:26.072655 20835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:26 dist-test-slave-ncp9 krb5kdc[20380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:26.075644 20835 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:14:26.077105 20835 webserver.cc:460] Webserver started at http://127.17.16.66:44905/ using document root <none> and password file <none>
I20260811 09:14:26.077744 20835 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:26.077852 20835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:26.078157 20835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:26.081059 20835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "c7b7aaf633ff4e258252b3b039fb48e4"
format_stamp: "Formatted at 2026-08-11 09:14:26 on dist-test-slave-ncp9"
server_key: "4c959f9fe523c5f5db486e7d133cf5b0"
server_key_iv: "476d720b551857d08d2755c8f3ae871c"
server_key_version: "encryptionkey@0"
I20260811 09:14:26.081805 20835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "c7b7aaf633ff4e258252b3b039fb48e4"
format_stamp: "Formatted at 2026-08-11 09:14:26 on dist-test-slave-ncp9"
server_key: "4c959f9fe523c5f5db486e7d133cf5b0"
server_key_iv: "476d720b551857d08d2755c8f3ae871c"
server_key_version: "encryptionkey@0"
I20260811 09:14:26.086668 20835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:14:26.089910 20864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:26.091166 20835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:14:26.091313 20835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "c7b7aaf633ff4e258252b3b039fb48e4"
format_stamp: "Formatted at 2026-08-11 09:14:26 on dist-test-slave-ncp9"
server_key: "4c959f9fe523c5f5db486e7d133cf5b0"
server_key_iv: "476d720b551857d08d2755c8f3ae871c"
server_key_version: "encryptionkey@0"
I20260811 09:14:26.091488 20835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:26.123426 20835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:26.127496 20835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:26.127779 20835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:26.130488 20835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:26.130631 20835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:26.130770 20835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:26.130892 20835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 09:14:26 dist-test-slave-ncp9 krb5kdc[20380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:26.153373 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.130966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:53809 (local address 127.17.16.126:39149)
0811 09:14:26.131117 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:26.131122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:26.132699 (+ 1577us) server_negotiation.cc:407] Connection header received
0811 09:14:26.134847 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:26.134853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:26.134936 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:26.135056 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:26.137358 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.138442 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.140950 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.141394 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.141519 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.146415 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:26.146422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:26.146427 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:26.146472 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:26.149692 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:26.150606 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:26.150612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:26.150615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:26.150771 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:26.151312 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:26.151317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:26.151321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:26.151618 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:26.151755 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:26.152742 (+ 987us) server_negotiation.cc:299] Negotiation successful
0811 09:14:26.153022 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 09:14:26.154359 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.131641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39149 (local address 127.17.16.66:53809)
0811 09:14:26.132517 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:26.132559 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:26.134528 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:26.135345 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:26.135358 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:26.136177 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:26.137160 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.137189 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.138622 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.138628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.140778 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.140794 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.141650 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.141658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.141878 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.142680 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:26.142703 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:26.146157 (+ 3454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:26.149818 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:26.149827 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:26.149848 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:26.150443 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:26.150898 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:26.150903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:26.150906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:26.151187 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:26.151748 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:26.151756 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:26.152240 (+ 484us) client_negotiation.cc:769] Sending connection context
0811 09:14:26.152619 (+ 379us) client_negotiation.cc:240] Negotiation successful
0811 09:14:26.153134 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":385,"threads_started":1}
I20260811 09:14:26.172578 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.159604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:39837 (local address 127.17.16.126:39149)
0811 09:14:26.159802 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:26.159809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:26.160515 (+ 706us) server_negotiation.cc:407] Connection header received
0811 09:14:26.160668 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:26.160674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:26.160751 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:26.160857 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:26.161803 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.162980 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.164494 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.164928 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.165054 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.167017 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:26.167022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:26.167025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:26.167054 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:26.169411 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:26.170219 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:26.170222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:26.170225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:26.170354 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:26.170855 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:26.170860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:26.170864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:26.171273 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:26.171429 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:26.172062 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 09:14:26.172267 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:14:26.172807 20982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.159835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39149 (local address 127.17.16.66:39837)
0811 09:14:26.160419 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:26.160435 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:26.160560 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:26.160930 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:26.160936 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:26.161392 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:26.161678 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.161687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.163122 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.163126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.164315 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.164339 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.165298 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.165305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.165507 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.166140 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:26.166157 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:26.166832 (+ 675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:26.169568 (+ 2736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:26.169574 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:26.169578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:26.170113 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:26.170511 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:26.170516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:26.170520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:26.170685 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:26.171567 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:26.171575 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:26.171796 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 09:14:26.171956 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 09:14:26.172382 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":305,"threads_started":1}
I20260811 09:14:26.185624 20835 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:38537
I20260811 09:14:26.185668 21064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:38537 every 8 connection(s)
I20260811 09:14:26.186832 20835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:14:26.188934 21065 heartbeater.cc:347] Connected to a master server at 127.17.16.126:39149
I20260811 09:14:26.189180 21065 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:26.189905 21065 heartbeater.cc:511] Master 127.17.16.126:39149 requested a full tablet report, sending...
I20260811 09:14:26.191259 20453 ts_manager.cc:194] Registered new tserver with Master: c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537)
I20260811 09:14:26.192170 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 20835
I20260811 09:14:26.192350 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:14:26.192308 20453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:53809
I20260811 09:14:26.192934 17473 external_mini_cluster.cc:1428] Setting key 66bfb5b5cf09efdff16244573916df9a
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:14:26.237561 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:39149
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:26.573959 21075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:26.574234 21075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:26.574376 21075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:26.581482 21075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:26.581596 21075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:26.581715 21075 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 09:14:26.581780 21075 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:14:26.581882 21075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:14:26.591617 21075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.16.126:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:26.593005 21075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:26.594349 21075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:26.607775 21084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:26.608003 21088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:26.608506 21083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:26.608495 21075 server_base.cc:1034] running on GCE node
I20260811 09:14:26.609261 21075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:26.610154 21075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:26.611384 21075 hybrid_clock.cc:648] HybridClock initialized: now 1786439666611333 us; error 35 us; skew 500 ppm
I20260811 09:14:26.611743 21075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:26 dist-test-slave-ncp9 krb5kdc[20380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:26.614921 21075 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:14:26.616477 21075 webserver.cc:460] Webserver started at http://127.17.16.67:45523/ using document root <none> and password file <none>
I20260811 09:14:26.617200 21075 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:26.617334 21075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:26.617668 21075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:26.620643 21075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "f6e08c6d062941da9238e8a48e5b7be1"
format_stamp: "Formatted at 2026-08-11 09:14:26 on dist-test-slave-ncp9"
server_key: "7df6f3a317de5b22879b49ce69eaeff1"
server_key_iv: "49daa66d9720beff0eae21eacee2584e"
server_key_version: "encryptionkey@0"
I20260811 09:14:26.621460 21075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "f6e08c6d062941da9238e8a48e5b7be1"
format_stamp: "Formatted at 2026-08-11 09:14:26 on dist-test-slave-ncp9"
server_key: "7df6f3a317de5b22879b49ce69eaeff1"
server_key_iv: "49daa66d9720beff0eae21eacee2584e"
server_key_version: "encryptionkey@0"
I20260811 09:14:26.626937 21075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:14:26.630505 21101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:26.631809 21075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:14:26.631973 21075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "f6e08c6d062941da9238e8a48e5b7be1"
format_stamp: "Formatted at 2026-08-11 09:14:26 on dist-test-slave-ncp9"
server_key: "7df6f3a317de5b22879b49ce69eaeff1"
server_key_iv: "49daa66d9720beff0eae21eacee2584e"
server_key_version: "encryptionkey@0"
I20260811 09:14:26.632190 21075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:26.660000 21075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:26.664083 21075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:26.664361 21075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:26.667011 21075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:26.667112 21075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:26.667232 21075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:26.667326 21075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:26.669565 20826 heartbeater.cc:503] Master 127.17.16.126:39149 was elected leader, sending a full tablet report...
Aug 11 09:14:26 dist-test-slave-ncp9 krb5kdc[20380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439666, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:26.689687 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.667404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:48373 (local address 127.17.16.126:39149)
0811 09:14:26.667615 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:26.667622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:26.668936 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 09:14:26.671375 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:26.671380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:26.671465 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:26.671565 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:26.673876 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.674801 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.677509 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.677970 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.678069 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.682342 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:26.682349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:26.682354 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:26.682398 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:26.685594 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:26.686506 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:26.686512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:26.686516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:26.686696 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:26.687356 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:26.687363 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:26.687367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:26.687760 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:26.687909 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:26.689144 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0811 09:14:26.689386 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:14:26.691110 21115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.667907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39149 (local address 127.17.16.67:48373)
0811 09:14:26.668762 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:26.668805 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:26.671099 (+ 2294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:26.671840 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:26.671857 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:26.672670 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:26.673674 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.673705 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.674959 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.674964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.677367 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.677384 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.678204 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.678211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.678493 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.679166 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:26.679196 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:26.682122 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:26.685720 (+ 3598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:26.685730 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:26.685745 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:26.686359 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:26.686867 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:26.686873 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:26.686877 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:26.687184 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:26.687991 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:26.688003 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:26.688555 (+ 552us) client_negotiation.cc:769] Sending connection context
0811 09:14:26.688956 (+ 401us) client_negotiation.cc:240] Negotiation successful
0811 09:14:26.689567 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":339,"threads_started":1}
I20260811 09:14:26.710800 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.697570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:60755 (local address 127.17.16.126:39149)
0811 09:14:26.697761 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:26.697768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:26.698338 (+ 570us) server_negotiation.cc:407] Connection header received
0811 09:14:26.698538 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:26.698544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:26.698640 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:26.698721 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:26.699703 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.700411 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.702001 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.702546 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.702669 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.704397 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:26.704402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:26.704405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:26.704443 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:26.707506 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:26.708576 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:26.708581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:26.708584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:26.708763 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:26.709235 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:26.709241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:26.709245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:26.709635 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:26.709774 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:26.710207 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 09:14:26.710429 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:14:26.710937 21218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.697595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39149 (local address 127.17.16.67:60755)
0811 09:14:26.698235 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:26.698260 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:26.698424 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:26.698765 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:26.698770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:26.699231 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:26.699572 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.699584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.700554 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.700559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.701865 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.701882 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.702735 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.702743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.702909 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.703478 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:26.703496 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:26.704210 (+ 714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:26.707688 (+ 3478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:26.707694 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:26.707699 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:26.708400 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:26.708941 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:26.708947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:26.708954 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:26.709120 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:26.709752 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:26.709757 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:26.709947 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 09:14:26.710092 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 09:14:26.710478 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":313,"threads_started":1}
I20260811 09:14:26.722613 21075 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:39643
I20260811 09:14:26.722646 21306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:39643 every 8 connection(s)
I20260811 09:14:26.723851 21075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:14:26.726055 21307 heartbeater.cc:347] Connected to a master server at 127.17.16.126:39149
I20260811 09:14:26.726307 21307 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:26.727025 21307 heartbeater.cc:511] Master 127.17.16.126:39149 requested a full tablet report, sending...
I20260811 09:14:26.728430 20453 ts_manager.cc:194] Registered new tserver with Master: f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643)
I20260811 09:14:26.729171 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 21075
I20260811 09:14:26.729341 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:14:26.729354 20453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:48373
I20260811 09:14:26.729812 17473 external_mini_cluster.cc:1428] Setting key 57dcd9893df47108adb163e443c0c5db
I20260811 09:14:26.732923 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:14:26 dist-test-slave-ncp9 krb5kdc[20380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:14:26 dist-test-slave-ncp9 krb5kdc[20380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:26.782840 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.770016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48508 (local address 127.17.16.126:39149)
0811 09:14:26.770257 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:26.770263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:26.770461 (+ 198us) server_negotiation.cc:407] Connection header received
0811 09:14:26.770619 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:26.770623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:26.770690 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:26.770774 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:26.772059 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.772771 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.773961 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.774380 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.774474 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.778096 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:26.778101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:26.778104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:26.778143 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:26.780441 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:26.781109 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:26.781113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:26.781115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:26.781246 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:26.781628 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:26.781631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:26.781633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:26.781897 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:26.782077 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:26.782313 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 09:14:26.782484 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:14:26.786800 20453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:14:26.793789 20453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:26.828404 21332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.806528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:38537 (local address 127.0.0.1:33830)
0811 09:14:26.820676 (+ 14148us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:26.820693 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:26.820880 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:26.821673 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:26.821676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:26.821697 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:26.822041 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.822052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.823608 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.823613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.825584 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.825599 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.827557 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.827563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.827780 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.827812 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:14:26.827893 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 09:14:26.828034 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14036,"mutex_wait_us":9,"thread_start_us":12530,"threads_started":1}
I20260811 09:14:26.828991 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.806709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48698 (local address 127.17.16.65:45449)
0811 09:14:26.817098 (+ 10389us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:26.817108 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:26.819467 (+ 2359us) server_negotiation.cc:407] Connection header received
0811 09:14:26.819817 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:26.819826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:26.820052 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:26.820252 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:26.821296 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.822746 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.825340 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.826720 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.826860 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.827348 (+ 488us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:26.828473 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0811 09:14:26.828658 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10217,"thread_start_us":9872,"threads_started":1}
I20260811 09:14:26.829082 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.807043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.17.16.66:38537)
0811 09:14:26.818871 (+ 11828us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:26.818879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:26.820781 (+ 1902us) server_negotiation.cc:407] Connection header received
0811 09:14:26.821118 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:26.821126 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:26.821310 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:26.821455 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:26.822231 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.823416 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.825758 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.827097 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.827230 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.827400 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:26.828670 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0811 09:14:26.828815 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11686,"thread_start_us":11554,"threads_started":1}
I20260811 09:14:26.828452 21336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.806152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45449 (local address 127.0.0.1:48698)
0811 09:14:26.819357 (+ 13205us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:26.819374 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:26.819570 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:26.820653 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:26.820663 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:26.820691 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:26.821105 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.821119 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.822915 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.822920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.825186 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.825203 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.826924 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.826931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.827512 (+ 581us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.827625 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 09:14:26.827752 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 09:14:26.827952 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13042,"mutex_wait_us":166,"thread_start_us":11100,"threads_started":1}
I20260811 09:14:26.829504 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.806273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.17.16.67:39643)
0811 09:14:26.818622 (+ 12349us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:26.818630 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:26.818665 (+ 35us) server_negotiation.cc:407] Connection header received
0811 09:14:26.818780 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:26.818787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:26.819045 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:26.819670 (+ 625us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:26.820846 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.822642 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.826371 (+ 3729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.827602 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.827725 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.827943 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:26.829064 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0811 09:14:26.829231 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12180,"thread_start_us":12005,"threads_started":1}
I20260811 09:14:26.828884 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.806099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:39643 (local address 127.0.0.1:52956)
0811 09:14:26.817692 (+ 11593us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:26.817751 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:26.817941 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:26.819295 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:26.819302 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:26.819333 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:26.820388 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.820403 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.823023 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.823028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.826211 (+ 3183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.826222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.828107 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.828114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.828293 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.828325 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:14:26.828427 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 09:14:26.828553 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11308,"mutex_wait_us":916,"thread_start_us":14279,"threads_started":1}
I20260811 09:14:26.832882 21181 tablet_service.cc:1467] Processing CreateTablet for tablet da88d987a427444299a908845c358798 (DEFAULT_TABLE table=test-table [id=613400a5617247d7b895a5cace149956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:26.832885 20705 tablet_service.cc:1467] Processing CreateTablet for tablet da88d987a427444299a908845c358798 (DEFAULT_TABLE table=test-table [id=613400a5617247d7b895a5cace149956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:26.834085 20705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da88d987a427444299a908845c358798. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:26.834091 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da88d987a427444299a908845c358798. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:26.839248 20947 tablet_service.cc:1467] Processing CreateTablet for tablet da88d987a427444299a908845c358798 (DEFAULT_TABLE table=test-table [id=613400a5617247d7b895a5cace149956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:26.840408 20947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da88d987a427444299a908845c358798. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:26.846685 21341 tablet_bootstrap.cc:492] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: Bootstrap starting.
I20260811 09:14:26.851272 21341 tablet_bootstrap.cc:654] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:26.852631 21341 log.cc:824] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:26.853534 21346 tablet_bootstrap.cc:492] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: Bootstrap starting.
I20260811 09:14:26.855248 21343 tablet_bootstrap.cc:492] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: Bootstrap starting.
I20260811 09:14:26.857192 21346 tablet_bootstrap.cc:654] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:26.858105 21346 log.cc:824] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:26.858623 21341 tablet_bootstrap.cc:492] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: No bootstrap required, opened a new log
I20260811 09:14:26.858769 21343 tablet_bootstrap.cc:654] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:26.858958 21341 ts_tablet_manager.cc:1397] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260811 09:14:26.860011 21343 log.cc:824] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:26.861650 21346 tablet_bootstrap.cc:492] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: No bootstrap required, opened a new log
I20260811 09:14:26.861884 21346 ts_tablet_manager.cc:1397] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 09:14:26.863035 21343 tablet_bootstrap.cc:492] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: No bootstrap required, opened a new log
I20260811 09:14:26.863317 21343 ts_tablet_manager.cc:1397] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 09:14:26.869143 21346 raft_consensus.cc:348] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } }
I20260811 09:14:26.869616 21346 raft_consensus.cc:374] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:26.869719 21346 raft_consensus.cc:729] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b7aaf633ff4e258252b3b039fb48e4, State: Initialized, Role: FOLLOWER
I20260811 09:14:26.869626 21341 raft_consensus.cc:348] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } }
I20260811 09:14:26.870086 21341 raft_consensus.cc:374] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:26.870190 21341 raft_consensus.cc:729] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e08c6d062941da9238e8a48e5b7be1, State: Initialized, Role: FOLLOWER
I20260811 09:14:26.870435 21346 consensus_queue.cc:260] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } }
I20260811 09:14:26.870884 21341 consensus_queue.cc:260] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } }
I20260811 09:14:26.871651 21065 heartbeater.cc:503] Master 127.17.16.126:39149 was elected leader, sending a full tablet report...
I20260811 09:14:26.872164 21307 heartbeater.cc:503] Master 127.17.16.126:39149 was elected leader, sending a full tablet report...
I20260811 09:14:26.872275 21346 ts_tablet_manager.cc:1428] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: Time spent starting tablet: real 0.010s user 0.001s sys 0.009s
I20260811 09:14:26.872879 21341 ts_tablet_manager.cc:1428] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260811 09:14:26.875581 21343 raft_consensus.cc:348] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } }
I20260811 09:14:26.876086 21343 raft_consensus.cc:374] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:26.876257 21343 raft_consensus.cc:729] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dee37677845496fb27c1256f148d60d, State: Initialized, Role: FOLLOWER
I20260811 09:14:26.876858 21343 consensus_queue.cc:260] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } }
I20260811 09:14:26.878607 21343 ts_tablet_manager.cc:1428] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260811 09:14:26.912420 20827 tablet.cc:2369] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:14:26.925753 21350 raft_consensus.cc:482] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:26.926055 21350 raft_consensus.cc:504] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } }
I20260811 09:14:26.927731 21350 leader_election.cc:288] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643), 4dee37677845496fb27c1256f148d60d (127.17.16.65:45449)
I20260811 09:14:26.934249 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.928194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:40295 (local address 127.17.16.67:39643)
0811 09:14:26.928411 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:26.928417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:26.929166 (+ 749us) server_negotiation.cc:407] Connection header received
0811 09:14:26.929330 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:26.929336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:26.929397 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:26.929472 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:26.930125 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.930827 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.932557 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.933318 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.933472 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.933559 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:26.933886 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 09:14:26.934004 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 09:14:26.934254 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.928189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:35165 (local address 127.17.16.65:45449)
0811 09:14:26.928397 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:26.928405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:26.929107 (+ 702us) server_negotiation.cc:407] Connection header received
0811 09:14:26.929316 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:26.929320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:26.929380 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:26.929453 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:26.930050 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.930781 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.932538 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.933342 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.933495 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.933566 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:26.933916 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 09:14:26.934037 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:14:26.934521 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.928095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45449 (local address 127.17.16.66:35165)
0811 09:14:26.928997 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:26.929013 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:26.929171 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:26.929480 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:26.929485 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:26.929508 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:26.929899 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.929911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.930902 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.930908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.932390 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.932402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.933558 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.933564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.933766 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.933790 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:14:26.933872 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 09:14:26.933992 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":40,"thread_start_us":514,"threads_started":1}
I20260811 09:14:26.934670 21357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:26.928176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:39643 (local address 127.17.16.66:40295)
0811 09:14:26.929050 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:26.929069 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:26.929195 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:26.929498 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:26.929502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:26.929524 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:26.929965 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.929975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.930957 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.930962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.932399 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:26.932410 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:26.933533 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:26.933543 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:26.933740 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:26.933768 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 09:14:26.933849 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 09:14:26.933970 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":34,"thread_start_us":438,"threads_started":1}
I20260811 09:14:26.935472 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da88d987a427444299a908845c358798" candidate_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e08c6d062941da9238e8a48e5b7be1" is_pre_election: true
I20260811 09:14:26.935470 20738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da88d987a427444299a908845c358798" candidate_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dee37677845496fb27c1256f148d60d" is_pre_election: true
I20260811 09:14:26.935952 21222 raft_consensus.cc:2454] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7b7aaf633ff4e258252b3b039fb48e4 in term 0.
I20260811 09:14:26.935966 20738 raft_consensus.cc:2454] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7b7aaf633ff4e258252b3b039fb48e4 in term 0.
I20260811 09:14:26.936558 20867 leader_election.cc:302] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dee37677845496fb27c1256f148d60d, c7b7aaf633ff4e258252b3b039fb48e4; no voters:
I20260811 09:14:26.936950 21350 raft_consensus.cc:2781] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:26.937124 21350 raft_consensus.cc:482] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:26.937242 21350 raft_consensus.cc:3037] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Advancing to term 1
W20260811 09:14:26.938650 21067 tablet.cc:2369] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:14:26.939486 21350 raft_consensus.cc:504] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } }
I20260811 09:14:26.940054 21350 leader_election.cc:288] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [CANDIDATE]: Term 1 election: Requested vote from peers f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643), 4dee37677845496fb27c1256f148d60d (127.17.16.65:45449)
I20260811 09:14:26.940421 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da88d987a427444299a908845c358798" candidate_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e08c6d062941da9238e8a48e5b7be1"
I20260811 09:14:26.940526 20738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da88d987a427444299a908845c358798" candidate_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dee37677845496fb27c1256f148d60d"
I20260811 09:14:26.940600 21222 raft_consensus.cc:3037] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:26.940711 20738 raft_consensus.cc:3037] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:26.943244 20738 raft_consensus.cc:2454] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7b7aaf633ff4e258252b3b039fb48e4 in term 1.
I20260811 09:14:26.943519 21222 raft_consensus.cc:2454] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7b7aaf633ff4e258252b3b039fb48e4 in term 1.
I20260811 09:14:26.943611 20867 leader_election.cc:302] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dee37677845496fb27c1256f148d60d, c7b7aaf633ff4e258252b3b039fb48e4; no voters:
I20260811 09:14:26.943903 21350 raft_consensus.cc:2781] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:26.944346 21350 raft_consensus.cc:686] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 LEADER]: Becoming Leader. State: Replica: c7b7aaf633ff4e258252b3b039fb48e4, State: Running, Role: LEADER
I20260811 09:14:26.944844 21350 consensus_queue.cc:237] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } }
I20260811 09:14:26.949793 20453 catalog_manager.cc:5495] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66). New cstate: current_term: 1 leader_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } health_report { overall_health: HEALTHY } } }
W20260811 09:14:26.975513 21311 tablet.cc:2369] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 09:14:27 dist-test-slave-ncp9 krb5kdc[20380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439664, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:27.015460 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.003399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39149 (local address 127.0.0.1:48512)
0811 09:14:27.003975 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:27.003990 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:27.004117 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:27.004398 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:27.004401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:27.005123 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:27.005413 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.005424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.006392 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.006396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.007343 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.007353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.008018 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.008022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.008152 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.008751 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:27.008768 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:27.010870 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:27.013391 (+ 2521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:27.013402 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:27.013408 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:27.013785 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:27.014096 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:27.014099 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:27.014102 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:27.014244 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:27.014688 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:27.014693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:27.014829 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 09:14:27.014957 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 09:14:27.015155 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":328,"threads_started":1}
I20260811 09:14:27.015498 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.003376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.17.16.126:39149)
0811 09:14:27.003549 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:27.003554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:27.004071 (+ 517us) server_negotiation.cc:407] Connection header received
0811 09:14:27.004222 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:27.004225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:27.004286 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:27.004364 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:27.005560 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.006283 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.007454 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.007869 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.007954 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.011034 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:27.011039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:27.011042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:27.011071 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:27.013287 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:27.013877 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:27.013881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:27.013883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:27.014001 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:27.014331 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:27.014334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:27.014336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:27.014588 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:27.014691 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:27.015056 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 09:14:27.015256 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":59}
I20260811 09:14:27.019979 20453 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:48512:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 09:14:27.020965 20453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:27.027256 20947 tablet_service.cc:1467] Processing CreateTablet for tablet 59945bc4583a4265aa2227b5bc2706b7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ae5c6f1d644d4e5b8861b69f9a05196f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:14:27.027065 20705 tablet_service.cc:1467] Processing CreateTablet for tablet 59945bc4583a4265aa2227b5bc2706b7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ae5c6f1d644d4e5b8861b69f9a05196f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:14:27.027415 21181 tablet_service.cc:1467] Processing CreateTablet for tablet 59945bc4583a4265aa2227b5bc2706b7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ae5c6f1d644d4e5b8861b69f9a05196f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:14:27.028081 20947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59945bc4583a4265aa2227b5bc2706b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:27.028093 20705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59945bc4583a4265aa2227b5bc2706b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:27.028301 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59945bc4583a4265aa2227b5bc2706b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:27.033968 21341 tablet_bootstrap.cc:492] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: Bootstrap starting.
I20260811 09:14:27.035075 21346 tablet_bootstrap.cc:492] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: Bootstrap starting.
I20260811 09:14:27.035508 21341 tablet_bootstrap.cc:654] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:27.035528 21343 tablet_bootstrap.cc:492] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: Bootstrap starting.
I20260811 09:14:27.036767 21346 tablet_bootstrap.cc:654] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:27.036943 21343 tablet_bootstrap.cc:654] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:27.038638 21341 tablet_bootstrap.cc:492] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: No bootstrap required, opened a new log
I20260811 09:14:27.038791 21341 ts_tablet_manager.cc:1397] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 09:14:27.038992 21346 tablet_bootstrap.cc:492] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: No bootstrap required, opened a new log
I20260811 09:14:27.039386 21346 ts_tablet_manager.cc:1397] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260811 09:14:27.039480 21341 raft_consensus.cc:348] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.039686 21341 raft_consensus.cc:374] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:27.039743 21343 tablet_bootstrap.cc:492] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: No bootstrap required, opened a new log
I20260811 09:14:27.039779 21341 raft_consensus.cc:729] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e08c6d062941da9238e8a48e5b7be1, State: Initialized, Role: FOLLOWER
I20260811 09:14:27.039882 21343 ts_tablet_manager.cc:1397] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 09:14:27.040005 21341 consensus_queue.cc:260] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.040143 21346 raft_consensus.cc:348] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.040364 21346 raft_consensus.cc:374] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:27.040460 21346 raft_consensus.cc:729] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b7aaf633ff4e258252b3b039fb48e4, State: Initialized, Role: FOLLOWER
I20260811 09:14:27.040421 21353 tablet_replica.cc:439] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } }
I20260811 09:14:27.040601 21343 raft_consensus.cc:348] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.040764 21353 tablet_replica.cc:442] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:27.040819 21343 raft_consensus.cc:374] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:27.040932 21343 raft_consensus.cc:729] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dee37677845496fb27c1256f148d60d, State: Initialized, Role: FOLLOWER
I20260811 09:14:27.040871 21346 consensus_queue.cc:260] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.041249 21341 ts_tablet_manager.cc:1428] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 09:14:27.041348 21350 tablet_replica.cc:439] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } }
I20260811 09:14:27.041517 21346 ts_tablet_manager.cc:1428] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 09:14:27.041611 21350 tablet_replica.cc:442] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:27.041150 21343 consensus_queue.cc:260] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.042740 21343 ts_tablet_manager.cc:1428] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 09:14:27.043015 21354 tablet_replica.cc:439] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } }
I20260811 09:14:27.043371 21354 tablet_replica.cc:442] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:27.091974 21350 raft_consensus.cc:482] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:27.092182 21350 raft_consensus.cc:504] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.092851 21350 leader_election.cc:288] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dee37677845496fb27c1256f148d60d (127.17.16.65:45449), f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643)
I20260811 09:14:27.093391 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59945bc4583a4265aa2227b5bc2706b7" candidate_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e08c6d062941da9238e8a48e5b7be1" is_pre_election: true
I20260811 09:14:27.093405 20738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59945bc4583a4265aa2227b5bc2706b7" candidate_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dee37677845496fb27c1256f148d60d" is_pre_election: true
I20260811 09:14:27.093637 21222 raft_consensus.cc:2454] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7b7aaf633ff4e258252b3b039fb48e4 in term 0.
I20260811 09:14:27.093675 20738 raft_consensus.cc:2454] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7b7aaf633ff4e258252b3b039fb48e4 in term 0.
I20260811 09:14:27.094022 20866 leader_election.cc:302] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7b7aaf633ff4e258252b3b039fb48e4, f6e08c6d062941da9238e8a48e5b7be1; no voters:
I20260811 09:14:27.094340 21350 raft_consensus.cc:2781] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:27.094429 21350 raft_consensus.cc:482] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:27.094532 21350 raft_consensus.cc:3037] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:27.096822 21350 raft_consensus.cc:504] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.097357 21350 leader_election.cc:288] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [CANDIDATE]: Term 1 election: Requested vote from peers 4dee37677845496fb27c1256f148d60d (127.17.16.65:45449), f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643)
I20260811 09:14:27.097769 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59945bc4583a4265aa2227b5bc2706b7" candidate_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e08c6d062941da9238e8a48e5b7be1"
I20260811 09:14:27.097949 21222 raft_consensus.cc:3037] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:27.099924 21222 raft_consensus.cc:2454] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7b7aaf633ff4e258252b3b039fb48e4 in term 1.
I20260811 09:14:27.100281 20866 leader_election.cc:302] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7b7aaf633ff4e258252b3b039fb48e4, f6e08c6d062941da9238e8a48e5b7be1; no voters:
I20260811 09:14:27.100271 20738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59945bc4583a4265aa2227b5bc2706b7" candidate_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dee37677845496fb27c1256f148d60d"
I20260811 09:14:27.100490 20738 raft_consensus.cc:3037] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:27.100505 21350 raft_consensus.cc:2781] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:27.100682 21350 raft_consensus.cc:686] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 LEADER]: Becoming Leader. State: Replica: c7b7aaf633ff4e258252b3b039fb48e4, State: Running, Role: LEADER
I20260811 09:14:27.100956 21350 consensus_queue.cc:237] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.102717 21358 tablet_replica.cc:439] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: TxnStatusTablet state changed. Reason: New leader c7b7aaf633ff4e258252b3b039fb48e4. Latest consensus state: current_term: 1 leader_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } }
I20260811 09:14:27.103111 21358 tablet_replica.cc:442] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:14:27.103320 20738 raft_consensus.cc:2454] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7b7aaf633ff4e258252b3b039fb48e4 in term 1.
I20260811 09:14:27.104125 20453 catalog_manager.cc:5495] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66). New cstate: current_term: 1 leader_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } health_report { overall_health: UNKNOWN } } }
I20260811 09:14:27.109298 21376 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 09:14:27.109557 21376 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 09:14:27.139675 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.133229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:38537 (local address 127.0.0.1:33832)
0811 09:14:27.133452 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:27.133468 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:27.133576 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:27.134155 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:27.134160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:27.134174 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:14:27.134375 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.134387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.135839 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.135843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.137038 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.137051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.138082 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.138087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.138245 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.138299 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:14:27.139265 (+ 966us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:14:27.139276 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 09:14:27.139322 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 09:14:27.139436 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 09:14:27.139907 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.133185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.17.16.66:38537)
0811 09:14:27.133807 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:27.133812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:27.133825 (+ 13us) server_negotiation.cc:407] Connection header received
0811 09:14:27.133894 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:27.133897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:27.133973 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:27.134091 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:27.134560 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.135706 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.137176 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.137919 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.138066 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.138495 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:27.139168 (+ 673us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:27.139330 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:27.139380 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 09:14:27.139463 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502}
I20260811 09:14:27.147322 21222 raft_consensus.cc:1261] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Refusing update from remote peer c7b7aaf633ff4e258252b3b039fb48e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:27.147322 20738 raft_consensus.cc:1261] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Refusing update from remote peer c7b7aaf633ff4e258252b3b039fb48e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:27.148368 21358 consensus_queue.cc:1035] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:27.148689 21350 consensus_queue.cc:1035] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:27.156847 21354 tablet_replica.cc:439] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: TxnStatusTablet state changed. Reason: New leader c7b7aaf633ff4e258252b3b039fb48e4. Latest consensus state: current_term: 1 leader_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } }
I20260811 09:14:27.157162 21354 tablet_replica.cc:442] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:27.160087 21379 mvcc.cc:205] Tried to move back new op lower bound from 7317256876625305600 to 7317256876448153600. Current Snapshot: MvccSnapshot[applied={T|T < 7317256876625305600}]
I20260811 09:14:27.161242 21353 tablet_replica.cc:439] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: TxnStatusTablet state changed. Reason: New leader c7b7aaf633ff4e258252b3b039fb48e4. Latest consensus state: current_term: 1 leader_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } }
I20260811 09:14:27.161558 21353 tablet_replica.cc:442] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:27.169034 21393 tablet_replica.cc:439] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } }
I20260811 09:14:27.169932 21393 tablet_replica.cc:442] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:14:27.171276 21354 tablet_replica.cc:439] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } }
I20260811 09:14:27.171499 21354 tablet_replica.cc:442] T 59945bc4583a4265aa2227b5bc2706b7 P 4dee37677845496fb27c1256f148d60d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:27.171705 21350 tablet_replica.cc:439] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } }
I20260811 09:14:27.171983 21350 tablet_replica.cc:442] T 59945bc4583a4265aa2227b5bc2706b7 P c7b7aaf633ff4e258252b3b039fb48e4: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:14:27.178464 21353 tablet_replica.cc:439] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } } }
I20260811 09:14:27.178714 21353 tablet_replica.cc:442] T 59945bc4583a4265aa2227b5bc2706b7 P f6e08c6d062941da9238e8a48e5b7be1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:14:27.187913 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.181954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33836 (local address 127.17.16.66:38537)
0811 09:14:27.182110 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:27.182117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:27.182292 (+ 175us) server_negotiation.cc:407] Connection header received
0811 09:14:27.182537 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:27.182542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:27.182628 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:27.182716 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:27.183107 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.183864 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.185777 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.186454 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.186625 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.187125 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:27.187293 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:27.187391 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:27.187515 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 09:14:27.187612 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 09:14:27.214891 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.209322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:39667 (local address 127.17.16.66:38537)
0811 09:14:27.209496 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:27.209500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:27.210124 (+ 624us) server_negotiation.cc:407] Connection header received
0811 09:14:27.210298 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:27.210301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:27.210355 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:27.210426 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:27.210971 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.211789 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.213242 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.213686 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.213787 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.214216 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:27.214317 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:27.214388 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:27.214518 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 09:14:27.214592 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:14:27.215016 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.209301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:38537 (local address 127.17.16.66:39667)
0811 09:14:27.209932 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:27.209949 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:27.210175 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:27.210461 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:27.210465 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:27.210475 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:14:27.210830 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.210840 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.211914 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.211919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.213120 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.213134 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.213854 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.213859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.213968 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.214018 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:14:27.214406 (+ 388us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:14:27.214420 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 09:14:27.214487 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 09:14:27.214627 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":363,"threads_started":1}
I20260811 09:14:27.223793 20738 raft_consensus.cc:1261] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Refusing update from remote peer c7b7aaf633ff4e258252b3b039fb48e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:27.223891 21222 raft_consensus.cc:1261] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Refusing update from remote peer c7b7aaf633ff4e258252b3b039fb48e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:27.224316 21358 consensus_queue.cc:1035] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:27.224608 21390 consensus_queue.cc:1035] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:27.233081 21381 mvcc.cc:205] Tried to move back new op lower bound from 7317256876944248832 to 7317256875810021376. Current Snapshot: MvccSnapshot[applied={T|T < 7317256876944248832}]
I20260811 09:14:27.235930 21380 mvcc.cc:205] Tried to move back new op lower bound from 7317256876944248832 to 7317256875810021376. Current Snapshot: MvccSnapshot[applied={T|T < 7317256876944248832}]
I20260811 09:14:27.283075 20452 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:27.283351 20452 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:27.286780 20452 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62: Sending DeleteTablet for 3 replicas of tablet da88d987a427444299a908845c358798
I20260811 09:14:27.287825 21181 tablet_service.cc:1514] Processing DeleteTablet for tablet da88d987a427444299a908845c358798 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:27 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:52956
I20260811 09:14:27.287968 20705 tablet_service.cc:1514] Processing DeleteTablet for tablet da88d987a427444299a908845c358798 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:27 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:48698
I20260811 09:14:27.288144 20945 tablet_service.cc:1514] Processing DeleteTablet for tablet da88d987a427444299a908845c358798 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:27 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:33830
I20260811 09:14:27.290221 20452 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:14:27.290931 20452 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:27.299338 21418 tablet_replica.cc:330] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: stopping tablet replica
I20260811 09:14:27.300239 21179 tablet_service.cc:1467] Processing CreateTablet for tablet f84c7e402eac4ec2bc3c5313443c3a1f (DEFAULT_TABLE table=test-table [id=aee7ac73be9343b3bd0d4e0c810c0e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:27.300585 21418 raft_consensus.cc:2229] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:14:27.300717 21179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84c7e402eac4ec2bc3c5313443c3a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:27.301122 21417 tablet_replica.cc:330] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: stopping tablet replica
I20260811 09:14:27.301594 21417 raft_consensus.cc:2229] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:27.302043 21417 raft_consensus.cc:2258] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:27.302322 21418 raft_consensus.cc:2258] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:27.304145 21417 ts_tablet_manager.cc:1905] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:27.304777 21418 ts_tablet_manager.cc:1905] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:27.305763 20705 tablet_service.cc:1467] Processing CreateTablet for tablet f84c7e402eac4ec2bc3c5313443c3a1f (DEFAULT_TABLE table=test-table [id=aee7ac73be9343b3bd0d4e0c810c0e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:27.306213 20705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84c7e402eac4ec2bc3c5313443c3a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:27.309682 20947 tablet_service.cc:1467] Processing CreateTablet for tablet f84c7e402eac4ec2bc3c5313443c3a1f (DEFAULT_TABLE table=test-table [id=aee7ac73be9343b3bd0d4e0c810c0e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:27.310086 20947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84c7e402eac4ec2bc3c5313443c3a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:27.311900 21417 ts_tablet_manager.cc:1918] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:14:27.312064 21341 tablet_bootstrap.cc:492] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1: Bootstrap starting.
I20260811 09:14:27.312157 21417 log.cc:1196] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/da88d987a427444299a908845c358798
I20260811 09:14:27.312798 21417 ts_tablet_manager.cc:1939] T da88d987a427444299a908845c358798 P 4dee37677845496fb27c1256f148d60d: Deleting consensus metadata
I20260811 09:14:27.313819 21341 tablet_bootstrap.cc:654] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:27.315053 20428 catalog_manager.cc:4841] TS 4dee37677845496fb27c1256f148d60d (127.17.16.65:45449): tablet da88d987a427444299a908845c358798 (table test-table [id=613400a5617247d7b895a5cace149956]) successfully deleted
I20260811 09:14:27.317703 21343 tablet_bootstrap.cc:492] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d: Bootstrap starting.
I20260811 09:14:27.319238 21343 tablet_bootstrap.cc:654] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:27.319440 21346 tablet_bootstrap.cc:492] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4: Bootstrap starting.
I20260811 09:14:27.321220 21346 tablet_bootstrap.cc:654] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:27.326547 21341 tablet_bootstrap.cc:492] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1: No bootstrap required, opened a new log
I20260811 09:14:27.326726 21341 ts_tablet_manager.cc:1397] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260811 09:14:27.327517 21341 raft_consensus.cc:348] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.327772 21341 raft_consensus.cc:374] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:27.327853 21341 raft_consensus.cc:729] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e08c6d062941da9238e8a48e5b7be1, State: Initialized, Role: FOLLOWER
I20260811 09:14:27.328156 21341 consensus_queue.cc:260] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.331400 21341 ts_tablet_manager.cc:1428] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 09:14:27.332839 21414 tablet_replica.cc:330] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: stopping tablet replica
I20260811 09:14:27.333338 21414 raft_consensus.cc:2229] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:27.333701 21414 raft_consensus.cc:2258] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:27.335173 21346 tablet_bootstrap.cc:492] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4: No bootstrap required, opened a new log
I20260811 09:14:27.335371 21346 ts_tablet_manager.cc:1397] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.002s
I20260811 09:14:27.336406 21346 raft_consensus.cc:348] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.336659 21346 raft_consensus.cc:374] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:27.336755 21346 raft_consensus.cc:729] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b7aaf633ff4e258252b3b039fb48e4, State: Initialized, Role: FOLLOWER
I20260811 09:14:27.337082 21343 tablet_bootstrap.cc:492] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d: No bootstrap required, opened a new log
I20260811 09:14:27.337107 21346 consensus_queue.cc:260] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.337809 21343 ts_tablet_manager.cc:1397] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.010s
I20260811 09:14:27.338002 21418 ts_tablet_manager.cc:1918] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:14:27.338181 21418 log.cc:1196] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/da88d987a427444299a908845c358798
I20260811 09:14:27.338585 21343 raft_consensus.cc:348] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.338758 21418 ts_tablet_manager.cc:1939] T da88d987a427444299a908845c358798 P c7b7aaf633ff4e258252b3b039fb48e4: Deleting consensus metadata
I20260811 09:14:27.338832 21343 raft_consensus.cc:374] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:27.338913 21343 raft_consensus.cc:729] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dee37677845496fb27c1256f148d60d, State: Initialized, Role: FOLLOWER
I20260811 09:14:27.339200 21343 consensus_queue.cc:260] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.340385 21346 ts_tablet_manager.cc:1428] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260811 09:14:27.340494 20429 catalog_manager.cc:4841] TS c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537): tablet da88d987a427444299a908845c358798 (table test-table [id=613400a5617247d7b895a5cace149956]) successfully deleted
I20260811 09:14:27.340651 21343 ts_tablet_manager.cc:1428] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 09:14:27.341379 21414 ts_tablet_manager.cc:1905] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:27.347249 21414 ts_tablet_manager.cc:1918] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:14:27.347393 21414 log.cc:1196] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/da88d987a427444299a908845c358798
I20260811 09:14:27.347803 21414 ts_tablet_manager.cc:1939] T da88d987a427444299a908845c358798 P f6e08c6d062941da9238e8a48e5b7be1: Deleting consensus metadata
I20260811 09:14:27.349308 20427 catalog_manager.cc:4841] TS f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643): tablet da88d987a427444299a908845c358798 (table test-table [id=613400a5617247d7b895a5cace149956]) successfully deleted
I20260811 09:14:27.372495 21354 raft_consensus.cc:482] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:27.372720 21354 raft_consensus.cc:504] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.374594 21354 leader_election.cc:288] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537), f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643)
I20260811 09:14:27.383805 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.375866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:51305 (local address 127.17.16.67:39643)
0811 09:14:27.376045 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:27.376051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:27.377030 (+ 979us) server_negotiation.cc:407] Connection header received
0811 09:14:27.377183 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:27.377188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:27.377262 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:27.377353 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:27.378140 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.378985 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.381560 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.382755 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.382916 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.382991 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:27.383358 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 09:14:27.383495 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:14:27.383895 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.375380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:39643 (local address 127.17.16.65:51305)
0811 09:14:27.376921 (+ 1541us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:27.376938 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:27.377056 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:27.377402 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:27.377407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:27.377443 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:27.377932 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.377947 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.379136 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.379142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.381370 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.381386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.382995 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.383003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.383179 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.383213 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:14:27.383315 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 09:14:27.383464 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1406,"mutex_wait_us":5,"thread_start_us":845,"threads_started":1}
I20260811 09:14:27.384284 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.375662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:49827 (local address 127.17.16.66:38537)
0811 09:14:27.375873 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:27.375881 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:27.376748 (+ 867us) server_negotiation.cc:407] Connection header received
0811 09:14:27.377145 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:27.377151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:27.377262 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:27.377331 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:27.378288 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.379362 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.381765 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.382786 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.382987 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.383077 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:27.383585 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 09:14:27.383730 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:14:27.384657 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c7e402eac4ec2bc3c5313443c3a1f" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e08c6d062941da9238e8a48e5b7be1" is_pre_election: true
I20260811 09:14:27.384972 21222 raft_consensus.cc:2454] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 0.
I20260811 09:14:27.385054 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.375125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:38537 (local address 127.17.16.65:49827)
0811 09:14:27.376620 (+ 1495us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:27.376643 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:27.376877 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:27.377551 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:27.377556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:27.377599 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:27.378056 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.378072 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.379519 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.379524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.381608 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.381620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.383178 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.383185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.383382 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.383417 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:14:27.384318 (+ 901us) client_negotiation.cc:240] Negotiation successful
0811 09:14:27.384676 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"thread_start_us":1468,"threads_started":1}
I20260811 09:14:27.385526 20621 leader_election.cc:302] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dee37677845496fb27c1256f148d60d, f6e08c6d062941da9238e8a48e5b7be1; no voters:
I20260811 09:14:27.386003 21354 raft_consensus.cc:2781] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:27.386147 21354 raft_consensus.cc:482] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:27.386246 21354 raft_consensus.cc:3037] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:27.386369 20989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c7e402eac4ec2bc3c5313443c3a1f" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" is_pre_election: true
I20260811 09:14:27.386829 20989 raft_consensus.cc:2454] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 0.
I20260811 09:14:27.389464 21354 raft_consensus.cc:504] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.390159 21354 leader_election.cc:288] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 election: Requested vote from peers c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537), f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643)
I20260811 09:14:27.390566 20989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c7e402eac4ec2bc3c5313443c3a1f" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b7aaf633ff4e258252b3b039fb48e4"
I20260811 09:14:27.390605 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84c7e402eac4ec2bc3c5313443c3a1f" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e08c6d062941da9238e8a48e5b7be1"
I20260811 09:14:27.390779 21222 raft_consensus.cc:3037] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:27.390791 20989 raft_consensus.cc:3037] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:27.392948 21222 raft_consensus.cc:2454] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 1.
I20260811 09:14:27.393335 20621 leader_election.cc:302] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dee37677845496fb27c1256f148d60d, f6e08c6d062941da9238e8a48e5b7be1; no voters:
I20260811 09:14:27.393627 21354 raft_consensus.cc:2781] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:27.394114 20989 raft_consensus.cc:2454] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 1.
I20260811 09:14:27.394173 21354 raft_consensus.cc:686] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 1 LEADER]: Becoming Leader. State: Replica: 4dee37677845496fb27c1256f148d60d, State: Running, Role: LEADER
I20260811 09:14:27.394586 21354 consensus_queue.cc:237] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:27.398072 20453 catalog_manager.cc:5495] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dee37677845496fb27c1256f148d60d (127.17.16.65). New cstate: current_term: 1 leader_uuid: "4dee37677845496fb27c1256f148d60d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } health_report { overall_health: UNKNOWN } } }
I20260811 09:14:27.421350 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.416180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48708 (local address 127.17.16.65:45449)
0811 09:14:27.416337 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:27.416343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:27.416435 (+ 92us) server_negotiation.cc:407] Connection header received
0811 09:14:27.416689 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:27.416693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:27.416773 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:27.416916 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:27.417315 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.418047 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.419300 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.419696 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.419801 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.420211 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:27.420732 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:27.420859 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:27.420981 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 09:14:27.421067 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:14:27.437692 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.431877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:52229 (local address 127.17.16.66:38537)
0811 09:14:27.432078 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:27.432083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:27.432744 (+ 661us) server_negotiation.cc:407] Connection header received
0811 09:14:27.432945 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:27.432949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:27.433020 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:27.433097 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:27.433585 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.434356 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.435856 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.436294 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.436429 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.436926 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:27.437054 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:27.437130 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:27.437271 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 09:14:27.437355 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:14:27.437788 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.431911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:38537 (local address 127.17.16.65:52229)
0811 09:14:27.432508 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:27.432529 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:27.432793 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:27.433127 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:27.433131 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:27.433143 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:14:27.433425 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.433434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.434496 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.434500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.435724 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.435736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.436463 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.436469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.436637 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.436689 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:14:27.437149 (+ 460us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:14:27.437161 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 09:14:27.437234 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 09:14:27.437361 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":333,"threads_started":1}
I20260811 09:14:27.450939 21222 raft_consensus.cc:1261] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Refusing update from remote peer 4dee37677845496fb27c1256f148d60d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:27.451643 21432 consensus_queue.cc:1035] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:27.452708 20989 raft_consensus.cc:1261] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Refusing update from remote peer 4dee37677845496fb27c1256f148d60d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:27.453572 21432 consensus_queue.cc:1035] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:27.481841 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.475691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:35977 (local address 127.17.16.65:45449)
0811 09:14:27.475844 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:27.475849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:27.475865 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:14:27.475923 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:27.475928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:27.476050 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:27.476131 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:27.477027 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.478044 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.479919 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.480476 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.480613 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.481066 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:27.481199 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:27.481289 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:27.481477 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 09:14:27.481561 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:14:27.482168 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:27.475155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45449 (local address 127.17.16.66:35977)
0811 09:14:27.475340 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:27.475360 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:27.475475 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:27.476312 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:27.476317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:27.476330 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:14:27.476563 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.476581 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.478183 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.478189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.479778 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:27.479795 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:27.480645 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:27.480651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:27.480820 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:27.480872 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:14:27.481313 (+ 441us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:14:27.481325 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 09:14:27.481454 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 09:14:27.481617 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:14:27.483534 21380 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 09:14:29.086752 20453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:29.086968 20453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:29.090809 20453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62: Sending DeleteTablet for 3 replicas of tablet f84c7e402eac4ec2bc3c5313443c3a1f
I20260811 09:14:29.091663 20705 tablet_service.cc:1514] Processing DeleteTablet for tablet f84c7e402eac4ec2bc3c5313443c3a1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:29 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:48698
I20260811 09:14:29.091693 21181 tablet_service.cc:1514] Processing DeleteTablet for tablet f84c7e402eac4ec2bc3c5313443c3a1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:29 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:52956
I20260811 09:14:29.092060 20947 tablet_service.cc:1514] Processing DeleteTablet for tablet f84c7e402eac4ec2bc3c5313443c3a1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:29 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:33830
I20260811 09:14:29.108506 21498 tablet_replica.cc:330] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4: stopping tablet replica
I20260811 09:14:29.108740 21498 raft_consensus.cc:2229] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:29.109023 21498 raft_consensus.cc:2258] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:29.110095 21498 ts_tablet_manager.cc:1905] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:29.113351 21496 tablet_replica.cc:330] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d: stopping tablet replica
I20260811 09:14:29.113551 21496 raft_consensus.cc:2229] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:14:29.113703 21497 tablet_replica.cc:330] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1: stopping tablet replica
I20260811 09:14:29.113903 21497 raft_consensus.cc:2229] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:29.114099 21496 raft_consensus.cc:2258] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:29.114128 21497 raft_consensus.cc:2258] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:29.115021 21497 ts_tablet_manager.cc:1905] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:29.117568 21496 ts_tablet_manager.cc:1905] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:29.118149 21498 ts_tablet_manager.cc:1918] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:14:29.118290 21498 log.cc:1196] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/f84c7e402eac4ec2bc3c5313443c3a1f
I20260811 09:14:29.118737 21498 ts_tablet_manager.cc:1939] T f84c7e402eac4ec2bc3c5313443c3a1f P c7b7aaf633ff4e258252b3b039fb48e4: Deleting consensus metadata
I20260811 09:14:29.119926 20429 catalog_manager.cc:4841] TS c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537): tablet f84c7e402eac4ec2bc3c5313443c3a1f (table test-table [id=aee7ac73be9343b3bd0d4e0c810c0e37]) successfully deleted
I20260811 09:14:29.121145 21497 ts_tablet_manager.cc:1918] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:14:29.121282 21497 log.cc:1196] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/f84c7e402eac4ec2bc3c5313443c3a1f
I20260811 09:14:29.121693 21497 ts_tablet_manager.cc:1939] T f84c7e402eac4ec2bc3c5313443c3a1f P f6e08c6d062941da9238e8a48e5b7be1: Deleting consensus metadata
I20260811 09:14:29.122798 20427 catalog_manager.cc:4841] TS f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643): tablet f84c7e402eac4ec2bc3c5313443c3a1f (table test-table [id=aee7ac73be9343b3bd0d4e0c810c0e37]) successfully deleted
I20260811 09:14:29.123823 21496 ts_tablet_manager.cc:1918] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:14:29.123934 21496 log.cc:1196] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/f84c7e402eac4ec2bc3c5313443c3a1f
I20260811 09:14:29.124258 21496 ts_tablet_manager.cc:1939] T f84c7e402eac4ec2bc3c5313443c3a1f P 4dee37677845496fb27c1256f148d60d: Deleting consensus metadata
I20260811 09:14:29.125131 20428 catalog_manager.cc:4841] TS 4dee37677845496fb27c1256f148d60d (127.17.16.65:45449): tablet f84c7e402eac4ec2bc3c5313443c3a1f (table test-table [id=aee7ac73be9343b3bd0d4e0c810c0e37]) successfully deleted
Aug 11 09:14:29 dist-test-slave-ncp9 krb5kdc[20380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439669, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:14:29 dist-test-slave-ncp9 krb5kdc[20380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439669, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:29.155453 21512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:29.141440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.17.16.126:39149)
0811 09:14:29.141996 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:29.142001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:29.142062 (+ 61us) server_negotiation.cc:407] Connection header received
0811 09:14:29.142256 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:29.142260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:29.142324 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:29.142406 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:29.143809 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:29.144603 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:29.145826 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:29.146240 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:29.146336 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:29.150161 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:29.150167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:29.150170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:29.150203 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:29.152806 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:29.153471 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:29.153476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:29.153478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:29.153601 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:29.153958 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:29.153961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:29.153964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:29.154319 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:29.154562 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:29.154880 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 09:14:29.155107 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":275,"threads_started":1}
I20260811 09:14:29.158746 20453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:14:29.159380 20453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:29.165678 20705 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1d046096c64339ba0d410d8675248a (DEFAULT_TABLE table=test-table [id=5c13e449c31349cfa139e11ccbfab44d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:29.165674 20947 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1d046096c64339ba0d410d8675248a (DEFAULT_TABLE table=test-table [id=5c13e449c31349cfa139e11ccbfab44d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:29.165903 21181 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1d046096c64339ba0d410d8675248a (DEFAULT_TABLE table=test-table [id=5c13e449c31349cfa139e11ccbfab44d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:29.166146 20705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1d046096c64339ba0d410d8675248a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:29.166183 20947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1d046096c64339ba0d410d8675248a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:29.166278 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1d046096c64339ba0d410d8675248a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:29.172160 21514 tablet_bootstrap.cc:492] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d: Bootstrap starting.
I20260811 09:14:29.172842 21515 tablet_bootstrap.cc:492] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1: Bootstrap starting.
I20260811 09:14:29.173444 21519 tablet_bootstrap.cc:492] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4: Bootstrap starting.
I20260811 09:14:29.174100 21514 tablet_bootstrap.cc:654] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:29.174486 21515 tablet_bootstrap.cc:654] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:29.174917 21519 tablet_bootstrap.cc:654] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:29.176735 21515 tablet_bootstrap.cc:492] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1: No bootstrap required, opened a new log
I20260811 09:14:29.176872 21515 ts_tablet_manager.cc:1397] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 09:14:29.177095 21519 tablet_bootstrap.cc:492] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4: No bootstrap required, opened a new log
I20260811 09:14:29.177287 21519 ts_tablet_manager.cc:1397] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 09:14:29.177446 21514 tablet_bootstrap.cc:492] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d: No bootstrap required, opened a new log
I20260811 09:14:29.177585 21514 ts_tablet_manager.cc:1397] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 09:14:29.177582 21515 raft_consensus.cc:348] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.177811 21515 raft_consensus.cc:374] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:29.177906 21515 raft_consensus.cc:729] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e08c6d062941da9238e8a48e5b7be1, State: Initialized, Role: FOLLOWER
I20260811 09:14:29.178141 21515 consensus_queue.cc:260] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.178098 21519 raft_consensus.cc:348] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.178335 21519 raft_consensus.cc:374] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:29.178359 21514 raft_consensus.cc:348] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.178498 21519 raft_consensus.cc:729] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b7aaf633ff4e258252b3b039fb48e4, State: Initialized, Role: FOLLOWER
I20260811 09:14:29.178602 21514 raft_consensus.cc:374] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:29.178701 21514 raft_consensus.cc:729] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dee37677845496fb27c1256f148d60d, State: Initialized, Role: FOLLOWER
I20260811 09:14:29.178879 21515 ts_tablet_manager.cc:1428] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 09:14:29.178934 21514 consensus_queue.cc:260] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.178805 21519 consensus_queue.cc:260] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.179209 21484 raft_consensus.cc:482] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:29.179489 21514 ts_tablet_manager.cc:1428] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 09:14:29.179363 21484 raft_consensus.cc:504] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.180099 21484 leader_election.cc:288] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537), f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643)
I20260811 09:14:29.180343 21519 ts_tablet_manager.cc:1428] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 09:14:29.181681 20988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1d046096c64339ba0d410d8675248a" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" is_pre_election: true
I20260811 09:14:29.181918 20988 raft_consensus.cc:2454] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 0.
I20260811 09:14:29.182367 20623 leader_election.cc:302] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dee37677845496fb27c1256f148d60d, c7b7aaf633ff4e258252b3b039fb48e4; no voters:
I20260811 09:14:29.182706 21484 raft_consensus.cc:2781] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:29.182723 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1d046096c64339ba0d410d8675248a" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e08c6d062941da9238e8a48e5b7be1" is_pre_election: true
I20260811 09:14:29.182860 21484 raft_consensus.cc:482] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:29.182930 21222 raft_consensus.cc:2454] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 0.
I20260811 09:14:29.182958 21484 raft_consensus.cc:3037] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:29.185160 21484 raft_consensus.cc:504] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.185720 21484 leader_election.cc:288] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 election: Requested vote from peers c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537), f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643)
I20260811 09:14:29.186095 20988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1d046096c64339ba0d410d8675248a" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b7aaf633ff4e258252b3b039fb48e4"
I20260811 09:14:29.186117 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1d046096c64339ba0d410d8675248a" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e08c6d062941da9238e8a48e5b7be1"
I20260811 09:14:29.186280 20988 raft_consensus.cc:3037] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:29.186295 21222 raft_consensus.cc:3037] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:29.188328 20988 raft_consensus.cc:2454] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 1.
I20260811 09:14:29.188653 20623 leader_election.cc:302] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dee37677845496fb27c1256f148d60d, c7b7aaf633ff4e258252b3b039fb48e4; no voters:
I20260811 09:14:29.188925 21484 raft_consensus.cc:2781] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:29.189065 21484 raft_consensus.cc:686] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 1 LEADER]: Becoming Leader. State: Replica: 4dee37677845496fb27c1256f148d60d, State: Running, Role: LEADER
I20260811 09:14:29.189199 21222 raft_consensus.cc:2454] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 1.
I20260811 09:14:29.189517 21484 consensus_queue.cc:237] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.192224 20451 catalog_manager.cc:5495] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dee37677845496fb27c1256f148d60d (127.17.16.65). New cstate: current_term: 1 leader_uuid: "4dee37677845496fb27c1256f148d60d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } health_report { overall_health: UNKNOWN } } }
I20260811 09:14:29.213908 21530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:29.208261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.17.16.65:45449)
0811 09:14:29.209055 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:29.209062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:29.209083 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:14:29.209141 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:29.209146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:29.209218 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:29.209318 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:29.209658 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:29.210811 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:29.212016 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:29.212658 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:29.212775 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:29.213167 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:29.213308 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:29.213393 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:29.213484 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 09:14:29.213574 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":516,"threads_started":1}
I20260811 09:14:29.222405 21222 raft_consensus.cc:1261] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Refusing update from remote peer 4dee37677845496fb27c1256f148d60d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:29.222446 20988 raft_consensus.cc:1261] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Refusing update from remote peer 4dee37677845496fb27c1256f148d60d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:29.222955 21484 consensus_queue.cc:1035] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:29.223227 21483 consensus_queue.cc:1035] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:29.265415 20451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:29.265653 20451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:29.268333 20451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62: Sending DeleteTablet for 3 replicas of tablet 7e1d046096c64339ba0d410d8675248a
I20260811 09:14:29.269129 20947 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1d046096c64339ba0d410d8675248a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:29 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:33830
I20260811 09:14:29.269116 20705 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1d046096c64339ba0d410d8675248a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:29 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:48698
I20260811 09:14:29.269249 21181 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1d046096c64339ba0d410d8675248a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:29 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:52956
I20260811 09:14:29.269411 21498 tablet_replica.cc:330] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4: stopping tablet replica
I20260811 09:14:29.269544 21497 tablet_replica.cc:330] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1: stopping tablet replica
I20260811 09:14:29.269567 21496 tablet_replica.cc:330] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d: stopping tablet replica
I20260811 09:14:29.269649 21498 raft_consensus.cc:2229] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:29.269724 21497 raft_consensus.cc:2229] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:29.269760 21496 raft_consensus.cc:2229] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:14:29.269832 21498 raft_consensus.cc:2258] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:29.269896 21497 raft_consensus.cc:2258] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:29.270088 21496 raft_consensus.cc:2258] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:29.270210 20451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:14:29.270841 20451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:29.271152 21498 ts_tablet_manager.cc:1905] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:29.271526 21496 ts_tablet_manager.cc:1905] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:29.272724 21497 ts_tablet_manager.cc:1905] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:29.276283 21498 ts_tablet_manager.cc:1918] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:14:29.276435 21498 log.cc:1196] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/7e1d046096c64339ba0d410d8675248a
I20260811 09:14:29.276865 21498 ts_tablet_manager.cc:1939] T 7e1d046096c64339ba0d410d8675248a P c7b7aaf633ff4e258252b3b039fb48e4: Deleting consensus metadata
I20260811 09:14:29.277897 21496 ts_tablet_manager.cc:1918] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:14:29.277915 20429 catalog_manager.cc:4841] TS c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537): tablet 7e1d046096c64339ba0d410d8675248a (table test-table [id=5c13e449c31349cfa139e11ccbfab44d]) successfully deleted
I20260811 09:14:29.278043 21496 log.cc:1196] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/7e1d046096c64339ba0d410d8675248a
I20260811 09:14:29.278465 21496 ts_tablet_manager.cc:1939] T 7e1d046096c64339ba0d410d8675248a P 4dee37677845496fb27c1256f148d60d: Deleting consensus metadata
I20260811 09:14:29.279114 21497 ts_tablet_manager.cc:1918] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:14:29.279264 21497 log.cc:1196] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/7e1d046096c64339ba0d410d8675248a
I20260811 09:14:29.279472 20428 catalog_manager.cc:4841] TS 4dee37677845496fb27c1256f148d60d (127.17.16.65:45449): tablet 7e1d046096c64339ba0d410d8675248a (table test-table [id=5c13e449c31349cfa139e11ccbfab44d]) successfully deleted
I20260811 09:14:29.279964 21497 ts_tablet_manager.cc:1939] T 7e1d046096c64339ba0d410d8675248a P f6e08c6d062941da9238e8a48e5b7be1: Deleting consensus metadata
I20260811 09:14:29.280845 20427 catalog_manager.cc:4841] TS f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643): tablet 7e1d046096c64339ba0d410d8675248a (table test-table [id=5c13e449c31349cfa139e11ccbfab44d]) successfully deleted
I20260811 09:14:29.282938 20705 tablet_service.cc:1467] Processing CreateTablet for tablet 74afea530a7b4438ada3ef86f3b93bd7 (DEFAULT_TABLE table=test-table [id=03968bd347a24728b9d99bba6169f5df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:29.283142 20947 tablet_service.cc:1467] Processing CreateTablet for tablet 74afea530a7b4438ada3ef86f3b93bd7 (DEFAULT_TABLE table=test-table [id=03968bd347a24728b9d99bba6169f5df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:29.283398 20705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74afea530a7b4438ada3ef86f3b93bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:29.283521 20947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74afea530a7b4438ada3ef86f3b93bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:29.285778 21181 tablet_service.cc:1467] Processing CreateTablet for tablet 74afea530a7b4438ada3ef86f3b93bd7 (DEFAULT_TABLE table=test-table [id=03968bd347a24728b9d99bba6169f5df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:29.286175 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74afea530a7b4438ada3ef86f3b93bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:29.288046 21514 tablet_bootstrap.cc:492] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d: Bootstrap starting.
I20260811 09:14:29.289682 21514 tablet_bootstrap.cc:654] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:29.290961 21515 tablet_bootstrap.cc:492] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1: Bootstrap starting.
I20260811 09:14:29.292397 21515 tablet_bootstrap.cc:654] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:29.293686 21519 tablet_bootstrap.cc:492] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4: Bootstrap starting.
I20260811 09:14:29.293722 21514 tablet_bootstrap.cc:492] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d: No bootstrap required, opened a new log
I20260811 09:14:29.293875 21514 ts_tablet_manager.cc:1397] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260811 09:14:29.294536 21514 raft_consensus.cc:348] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.294720 21514 raft_consensus.cc:374] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:29.294798 21514 raft_consensus.cc:729] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dee37677845496fb27c1256f148d60d, State: Initialized, Role: FOLLOWER
I20260811 09:14:29.295100 21519 tablet_bootstrap.cc:654] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:29.295027 21514 consensus_queue.cc:260] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.295599 21514 ts_tablet_manager.cc:1428] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 09:14:29.297024 21515 tablet_bootstrap.cc:492] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1: No bootstrap required, opened a new log
I20260811 09:14:29.297268 21515 ts_tablet_manager.cc:1397] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260811 09:14:29.298074 21515 raft_consensus.cc:348] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.298367 21515 raft_consensus.cc:374] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:29.298445 21515 raft_consensus.cc:729] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e08c6d062941da9238e8a48e5b7be1, State: Initialized, Role: FOLLOWER
I20260811 09:14:29.298693 21515 consensus_queue.cc:260] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.299216 21515 ts_tablet_manager.cc:1428] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 09:14:29.300549 21519 tablet_bootstrap.cc:492] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4: No bootstrap required, opened a new log
I20260811 09:14:29.300706 21519 ts_tablet_manager.cc:1397] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260811 09:14:29.301412 21519 raft_consensus.cc:348] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.301641 21519 raft_consensus.cc:374] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:29.301725 21519 raft_consensus.cc:729] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b7aaf633ff4e258252b3b039fb48e4, State: Initialized, Role: FOLLOWER
I20260811 09:14:29.301994 21519 consensus_queue.cc:260] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.302595 21519 ts_tablet_manager.cc:1428] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 09:14:29.450775 21484 raft_consensus.cc:482] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:29.450975 21484 raft_consensus.cc:504] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.451577 21484 leader_election.cc:288] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537), f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643)
I20260811 09:14:29.452129 20988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74afea530a7b4438ada3ef86f3b93bd7" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" is_pre_election: true
I20260811 09:14:29.452147 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74afea530a7b4438ada3ef86f3b93bd7" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e08c6d062941da9238e8a48e5b7be1" is_pre_election: true
I20260811 09:14:29.452373 21222 raft_consensus.cc:2454] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 0.
I20260811 09:14:29.452382 20988 raft_consensus.cc:2454] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 0.
I20260811 09:14:29.452812 20621 leader_election.cc:302] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4dee37677845496fb27c1256f148d60d, c7b7aaf633ff4e258252b3b039fb48e4, f6e08c6d062941da9238e8a48e5b7be1; no voters:
I20260811 09:14:29.453099 21484 raft_consensus.cc:2781] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:29.453214 21484 raft_consensus.cc:482] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:29.453285 21484 raft_consensus.cc:3037] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:29.455266 21484 raft_consensus.cc:504] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.455734 21484 leader_election.cc:288] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 election: Requested vote from peers c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537), f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643)
I20260811 09:14:29.456235 20988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74afea530a7b4438ada3ef86f3b93bd7" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b7aaf633ff4e258252b3b039fb48e4"
I20260811 09:14:29.456400 20988 raft_consensus.cc:3037] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:29.456382 21222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74afea530a7b4438ada3ef86f3b93bd7" candidate_uuid: "4dee37677845496fb27c1256f148d60d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e08c6d062941da9238e8a48e5b7be1"
I20260811 09:14:29.456580 21222 raft_consensus.cc:3037] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:29.458490 20988 raft_consensus.cc:2454] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 1.
I20260811 09:14:29.458643 21222 raft_consensus.cc:2454] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dee37677845496fb27c1256f148d60d in term 1.
I20260811 09:14:29.458909 20623 leader_election.cc:302] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dee37677845496fb27c1256f148d60d, c7b7aaf633ff4e258252b3b039fb48e4; no voters:
I20260811 09:14:29.459165 21484 raft_consensus.cc:2781] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:29.459342 21484 raft_consensus.cc:686] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 1 LEADER]: Becoming Leader. State: Replica: 4dee37677845496fb27c1256f148d60d, State: Running, Role: LEADER
I20260811 09:14:29.459623 21484 consensus_queue.cc:237] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } }
I20260811 09:14:29.462154 20451 catalog_manager.cc:5495] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dee37677845496fb27c1256f148d60d (127.17.16.65). New cstate: current_term: 1 leader_uuid: "4dee37677845496fb27c1256f148d60d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dee37677845496fb27c1256f148d60d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 } health_report { overall_health: UNKNOWN } } }
I20260811 09:14:29.493609 21222 raft_consensus.cc:1261] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Refusing update from remote peer 4dee37677845496fb27c1256f148d60d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:29.493638 20988 raft_consensus.cc:1261] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Refusing update from remote peer 4dee37677845496fb27c1256f148d60d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:29.494171 21484 consensus_queue.cc:1035] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e08c6d062941da9238e8a48e5b7be1" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 39643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:29.494448 21432 consensus_queue.cc:1035] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7b7aaf633ff4e258252b3b039fb48e4" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 09:14:29.513360 21464 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 09:14:31.096939 20453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:31.097244 20453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:31.101275 20453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ca67e87a37f4d008389631f1af75a62: Sending DeleteTablet for 3 replicas of tablet 74afea530a7b4438ada3ef86f3b93bd7
I20260811 09:14:31.102041 20705 tablet_service.cc:1514] Processing DeleteTablet for tablet 74afea530a7b4438ada3ef86f3b93bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:31 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:48698
I20260811 09:14:31.102092 20947 tablet_service.cc:1514] Processing DeleteTablet for tablet 74afea530a7b4438ada3ef86f3b93bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:31 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:33830
I20260811 09:14:31.102327 21181 tablet_service.cc:1514] Processing DeleteTablet for tablet 74afea530a7b4438ada3ef86f3b93bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:31 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:52956
I20260811 09:14:31.102828 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20584
I20260811 09:14:31.114596 21585 tablet_replica.cc:330] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1: stopping tablet replica
I20260811 09:14:31.114830 21585 raft_consensus.cc:2229] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:31.115114 21585 raft_consensus.cc:2258] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:31.115749 21584 tablet_replica.cc:330] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4: stopping tablet replica
I20260811 09:14:31.115954 21584 raft_consensus.cc:2229] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:31.116220 21584 raft_consensus.cc:2258] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:31.117096 21585 ts_tablet_manager.cc:1905] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:31.117866 21583 tablet_replica.cc:330] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d: stopping tablet replica
I20260811 09:14:31.118172 21584 ts_tablet_manager.cc:1905] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:31.118160 21583 raft_consensus.cc:2229] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:14:31.118702 21583 raft_consensus.cc:2258] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:31.119710 21583 ts_tablet_manager.cc:1905] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:31.123646 21585 ts_tablet_manager.cc:1918] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:14:31.123793 21585 log.cc:1196] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/74afea530a7b4438ada3ef86f3b93bd7
I20260811 09:14:31.124245 21585 ts_tablet_manager.cc:1939] T 74afea530a7b4438ada3ef86f3b93bd7 P f6e08c6d062941da9238e8a48e5b7be1: Deleting consensus metadata
I20260811 09:14:31.124629 21584 ts_tablet_manager.cc:1918] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:14:31.124764 21584 log.cc:1196] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/74afea530a7b4438ada3ef86f3b93bd7
I20260811 09:14:31.125290 21584 ts_tablet_manager.cc:1939] T 74afea530a7b4438ada3ef86f3b93bd7 P c7b7aaf633ff4e258252b3b039fb48e4: Deleting consensus metadata
I20260811 09:14:31.125505 20427 catalog_manager.cc:4841] TS f6e08c6d062941da9238e8a48e5b7be1 (127.17.16.67:39643): tablet 74afea530a7b4438ada3ef86f3b93bd7 (table test-table [id=03968bd347a24728b9d99bba6169f5df]) successfully deleted
Aug 11 09:14:31 dist-test-slave-ncp9 krb5kdc[20380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439669, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:14:31.126096 21583 ts_tablet_manager.cc:1918] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:14:31.126226 21583 log.cc:1196] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/74afea530a7b4438ada3ef86f3b93bd7
I20260811 09:14:31.126420 20429 catalog_manager.cc:4841] TS c7b7aaf633ff4e258252b3b039fb48e4 (127.17.16.66:38537): tablet 74afea530a7b4438ada3ef86f3b93bd7 (table test-table [id=03968bd347a24728b9d99bba6169f5df]) successfully deleted
I20260811 09:14:31.126677 21583 ts_tablet_manager.cc:1939] T 74afea530a7b4438ada3ef86f3b93bd7 P 4dee37677845496fb27c1256f148d60d: Deleting consensus metadata
I20260811 09:14:31.127619 20428 catalog_manager.cc:4841] TS 4dee37677845496fb27c1256f148d60d (127.17.16.65:45449): tablet 74afea530a7b4438ada3ef86f3b93bd7 (table test-table [id=03968bd347a24728b9d99bba6169f5df]) successfully deleted
I20260811 09:14:31.132184 21587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:31.104587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.17.16.65:45449)
0811 09:14:31.113678 (+ 9091us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:31.113686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:31.114516 (+ 830us) server_negotiation.cc:407] Connection header received
0811 09:14:31.114601 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:31.114606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:31.114762 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:31.114869 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:31.117271 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:31.118688 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:31.120610 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:31.121291 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:31.121448 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:31.127100 (+ 5652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:31.127116 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:31.127135 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:31.127166 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:31.129897 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:31.130521 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:31.130529 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:31.130531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:31.130656 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:31.130932 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:31.130935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:31.130937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:31.131265 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:31.131540 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:31.131730 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 09:14:31.131889 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8919,"thread_start_us":9107,"threads_started":1}
W20260811 09:14:31.132810 20812 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35804
W20260811 09:14:31.133421 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20584: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:14:31.133651 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 20584
I20260811 09:14:31.170953 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20835
Aug 11 09:14:31 dist-test-slave-ncp9 krb5kdc[20380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439669, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:14:31.185400 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:31.171744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.17.16.66:38537)
0811 09:14:31.172229 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:31.172234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:31.172251 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:14:31.172306 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:31.172310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:31.172373 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:31.172461 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:31.173701 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:31.174453 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:31.175605 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:31.175994 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:31.176097 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:31.180312 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:31.180327 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:31.180344 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:31.180378 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:31.183129 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:31.183722 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:31.183729 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:31.183731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:31.183874 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:31.184133 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:31.184146 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:31.184148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:31.184474 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:31.184699 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:31.184874 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 09:14:31.185100 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":257,"threads_started":1}
W20260811 09:14:31.186172 21054 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46580
W20260811 09:14:31.186815 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20835: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:14:31.186954 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 20835
I20260811 09:14:31.229058 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21075
Aug 11 09:14:31 dist-test-slave-ncp9 krb5kdc[20380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439669, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:14:31.243857 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:31.229802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.17.16.67:39643)
0811 09:14:31.230308 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:31.230313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:31.230329 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:14:31.230389 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:31.230393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:31.230454 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:31.230542 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:31.231759 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:31.232515 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:31.233816 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:31.234202 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:31.234294 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:31.238512 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:31.238566 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:31.238583 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:31.238624 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:31.241616 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:31.242250 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:31.242257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:31.242259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:31.242397 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:31.242676 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:31.242679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:31.242681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:31.243008 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:31.243233 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:31.243425 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 09:14:31.243591 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":279,"threads_started":1}
W20260811 09:14:31.244519 21291 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56510
W20260811 09:14:31.245193 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21075: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:14:31.245350 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 21075
I20260811 09:14:31.269352 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20403
I20260811 09:14:31.270545 20533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:31.387753 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 20403
2026-08-11T09:14:31Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8574 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:14:31 dist-test-slave-ncp9 krb5kdc[21595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:14:31 dist-test-slave-ncp9 krb5kdc[21595](info): set up 2 sockets
Aug 11 09:14:31 dist-test-slave-ncp9 krb5kdc[21595](info): commencing operation
krb5kdc: starting...
W20260811 09:14:32.615589 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.011s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:14:32 dist-test-slave-ncp9 krb5kdc[21595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:14:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:14:32.828435 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41017
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41017
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:33.152844 21613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:33.153162 21613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:33.153306 21613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:33.160475 21613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:33.160589 21613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:33.160668 21613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:33.160751 21613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:33.160821 21613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:33.170463 21613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:33.171749 21613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:33.173026 21613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:33.184398 21621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:33.184417 21625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:33.184422 21622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:33.185354 21613 server_base.cc:1034] running on GCE node
I20260811 09:14:33.186095 21613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:33.187440 21613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:33.188658 21613 hybrid_clock.cc:648] HybridClock initialized: now 1786439673188595 us; error 47 us; skew 500 ppm
I20260811 09:14:33.189028 21613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:33 dist-test-slave-ncp9 krb5kdc[21595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439673, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:33.192067 21613 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:14:33.193585 21613 webserver.cc:460] Webserver started at http://127.17.16.126:43649/ using document root <none> and password file <none>
I20260811 09:14:33.194242 21613 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:33.194358 21613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:33.194662 21613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:33.197685 21613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "414a99a67c6344a69d5ed5764c5ea3eb"
format_stamp: "Formatted at 2026-08-11 09:14:33 on dist-test-slave-ncp9"
server_key: "1746f07095b4736bb7be43c4f917cef5"
server_key_iv: "9b1e5de56ac0006ec995e2b7e3250873"
server_key_version: "encryptionkey@0"
I20260811 09:14:33.198431 21613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "414a99a67c6344a69d5ed5764c5ea3eb"
format_stamp: "Formatted at 2026-08-11 09:14:33 on dist-test-slave-ncp9"
server_key: "1746f07095b4736bb7be43c4f917cef5"
server_key_iv: "9b1e5de56ac0006ec995e2b7e3250873"
server_key_version: "encryptionkey@0"
I20260811 09:14:33.203679 21613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:14:33.206957 21637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:33.208263 21613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:14:33.208427 21613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "414a99a67c6344a69d5ed5764c5ea3eb"
format_stamp: "Formatted at 2026-08-11 09:14:33 on dist-test-slave-ncp9"
server_key: "1746f07095b4736bb7be43c4f917cef5"
server_key_iv: "9b1e5de56ac0006ec995e2b7e3250873"
server_key_version: "encryptionkey@0"
I20260811 09:14:33.208599 21613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:33.220108 21613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:33.224086 21613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:33.224334 21613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:33.242070 21613 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:41017
I20260811 09:14:33.242089 21740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:41017 every 8 connection(s)
I20260811 09:14:33.243470 21613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:33.247171 21743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:33.248854 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 21613
I20260811 09:14:33.249114 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:33.249928 17473 external_mini_cluster.cc:1428] Setting key 3d6cda5abf9e59419d9469eed33de4df
I20260811 09:14:33.259476 21743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb: Bootstrap starting.
I20260811 09:14:33.263073 21743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:33.264310 21743 log.cc:824] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:14:33 dist-test-slave-ncp9 krb5kdc[21595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:33.267076 21743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb: No bootstrap required, opened a new log
I20260811 09:14:33.272830 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:33.252005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.17.16.126:41017)
0811 09:14:33.252794 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:33.252805 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:33.252849 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:14:33.254616 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:33.254646 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:33.255000 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:33.255451 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:33.257335 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:33.258676 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:33.260362 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:33.261096 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:33.261221 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:33.265924 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:33.265932 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:33.265946 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:33.265979 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:33.268928 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:33.269581 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:33.269588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:33.269597 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:33.269780 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:33.270165 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:33.270169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:33.270171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:33.270673 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:33.270968 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:33.271437 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 09:14:33.271841 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":376,"threads_started":1}
I20260811 09:14:33.274647 21743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414a99a67c6344a69d5ed5764c5ea3eb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41017 } }
I20260811 09:14:33.274987 21743 raft_consensus.cc:374] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:33.275098 21743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 414a99a67c6344a69d5ed5764c5ea3eb, State: Initialized, Role: FOLLOWER
I20260811 09:14:33.275681 21743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414a99a67c6344a69d5ed5764c5ea3eb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41017 } }
I20260811 09:14:33.275946 21743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:33.276131 21743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:33.276307 21743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:33.278697 21743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414a99a67c6344a69d5ed5764c5ea3eb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41017 } }
I20260811 09:14:33.279207 21743 leader_election.cc:302] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 414a99a67c6344a69d5ed5764c5ea3eb; no voters:
I20260811 09:14:33.279731 21743 leader_election.cc:288] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:33.279850 21749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:33.280479 21749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [term 1 LEADER]: Becoming Leader. State: Replica: 414a99a67c6344a69d5ed5764c5ea3eb, State: Running, Role: LEADER
I20260811 09:14:33.281041 21749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414a99a67c6344a69d5ed5764c5ea3eb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41017 } }
I20260811 09:14:33.281605 21743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:33.284195 21753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 414a99a67c6344a69d5ed5764c5ea3eb. Latest consensus state: current_term: 1 leader_uuid: "414a99a67c6344a69d5ed5764c5ea3eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414a99a67c6344a69d5ed5764c5ea3eb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41017 } } }
I20260811 09:14:33.284588 21753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:33.283675 21750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "414a99a67c6344a69d5ed5764c5ea3eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "414a99a67c6344a69d5ed5764c5ea3eb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41017 } } }
I20260811 09:14:33.285019 21750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:33.286824 21765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:33.291816 21765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:33.300503 21765 catalog_manager.cc:1347] Generated new cluster ID: bf66525c37a04e8bb56442556110f951
I20260811 09:14:33.300623 21765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:33.316025 21765 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:33.317845 21765 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:33.328369 21765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 414a99a67c6344a69d5ed5764c5ea3eb: Generated new TSK 0
I20260811 09:14:33.329406 21765 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:14:33.386683 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41017
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:33.746827 21788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:33.747126 21788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:33.747248 21788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:33.754510 21788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:33.754634 21788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:33.754782 21788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:14:33.764770 21788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:33.766122 21788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:33.767920 21788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:33.781049 21799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:33.780977 21803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:33.781019 21800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:33.782224 21788 server_base.cc:1034] running on GCE node
I20260811 09:14:33.782850 21788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:33.783789 21788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:33.784989 21788 hybrid_clock.cc:648] HybridClock initialized: now 1786439673784935 us; error 44 us; skew 500 ppm
I20260811 09:14:33.785295 21788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:33 dist-test-slave-ncp9 krb5kdc[21595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439673, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:33.788496 21788 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:14:33.790020 21788 webserver.cc:460] Webserver started at http://127.17.16.65:38593/ using document root <none> and password file <none>
I20260811 09:14:33.790683 21788 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:33.790794 21788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:33.791100 21788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:33.794070 21788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "8f1edbd677d2415fa30a62a8db00ed59"
format_stamp: "Formatted at 2026-08-11 09:14:33 on dist-test-slave-ncp9"
server_key: "4338670466552590161120c4c847376d"
server_key_iv: "03e32dcadba23343b16832602667f0ae"
server_key_version: "encryptionkey@0"
I20260811 09:14:33.794935 21788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "8f1edbd677d2415fa30a62a8db00ed59"
format_stamp: "Formatted at 2026-08-11 09:14:33 on dist-test-slave-ncp9"
server_key: "4338670466552590161120c4c847376d"
server_key_iv: "03e32dcadba23343b16832602667f0ae"
server_key_version: "encryptionkey@0"
I20260811 09:14:33.799973 21788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:14:33.803261 21816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:33.804443 21788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 09:14:33.804600 21788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "8f1edbd677d2415fa30a62a8db00ed59"
format_stamp: "Formatted at 2026-08-11 09:14:33 on dist-test-slave-ncp9"
server_key: "4338670466552590161120c4c847376d"
server_key_iv: "03e32dcadba23343b16832602667f0ae"
server_key_version: "encryptionkey@0"
I20260811 09:14:33.804776 21788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:33.826393 21788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:33.830504 21788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:33.830768 21788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:33.831696 21788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:33.833071 21788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:33.833158 21788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:33.833274 21788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:33.833328 21788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:33.863580 21788 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:43683
I20260811 09:14:33.863618 22032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:43683 every 8 connection(s)
I20260811 09:14:33.864801 21788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:14:33.868142 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 21788
I20260811 09:14:33.868386 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:14:33.869182 17473 external_mini_cluster.cc:1428] Setting key 69124d2e4c7f0fba3c3b0aeee26d1d47
Aug 11 09:14:33 dist-test-slave-ncp9 krb5kdc[21595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439673, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:33.884521 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:33.867301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:43791 (local address 127.17.16.126:41017)
0811 09:14:33.867763 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:33.867767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:33.868824 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 09:14:33.870821 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:33.870827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:33.870905 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:33.871000 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:33.872497 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:33.873297 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:33.874803 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:33.875204 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:33.875323 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:33.878944 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:33.878949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:33.878953 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:33.878985 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:33.881482 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:33.882360 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:33.882364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:33.882366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:33.882480 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:33.882944 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:33.882948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:33.882950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:33.883218 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:33.883328 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:33.884099 (+ 771us) server_negotiation.cc:299] Negotiation successful
0811 09:14:33.884265 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":227,"threads_started":1}
I20260811 09:14:33.885917 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:33.867844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41017 (local address 127.17.16.65:43791)
0811 09:14:33.868666 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:33.868715 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:33.870550 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:33.871261 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:33.871269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:33.871774 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:33.872315 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:33.872332 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:33.873448 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:33.873452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:33.874653 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:33.874665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:33.875434 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:33.875441 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:33.875662 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:33.876297 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:33.876319 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:33.878709 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:33.881609 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:33.881618 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:33.881633 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:33.882253 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:33.882604 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:33.882608 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:33.882612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:33.882841 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:33.883352 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:33.883359 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:33.883740 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 09:14:33.884032 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 09:14:33.884518 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":352,"threads_started":1}
I20260811 09:14:33.887548 22036 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41017
I20260811 09:14:33.887852 22036 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:33.888545 22036 heartbeater.cc:511] Master 127.17.16.126:41017 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
I20260811 09:14:33.890617 21668 ts_manager.cc:194] Registered new tserver with Master: 8f1edbd677d2415fa30a62a8db00ed59 (127.17.16.65:43683)
I20260811 09:14:33.892050 21668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:43791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:14:33.920378 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41017
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:34.264142 22048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:34.264536 22048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:34.264695 22048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:34.272807 22048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:34.272969 22048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:34.273105 22048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:14:34.283696 22048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:34.285151 22048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:34.286494 22048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:34.299314 22068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:34.299429 22065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:34.299480 22064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:34.300293 22048 server_base.cc:1034] running on GCE node
I20260811 09:14:34.301062 22048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:34.302121 22048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:34.303354 22048 hybrid_clock.cc:648] HybridClock initialized: now 1786439674303308 us; error 36 us; skew 500 ppm
I20260811 09:14:34.303681 22048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:34 dist-test-slave-ncp9 krb5kdc[21595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439674, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:34.306879 22048 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:14:34.308365 22048 webserver.cc:460] Webserver started at http://127.17.16.66:40171/ using document root <none> and password file <none>
I20260811 09:14:34.309122 22048 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:34.309232 22048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:34.309579 22048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:34.312574 22048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "bf26b3af800a44058cdc89076eab1a2e"
format_stamp: "Formatted at 2026-08-11 09:14:34 on dist-test-slave-ncp9"
server_key: "f5ccb7e44105fd771797fa060c095930"
server_key_iv: "5cd6618d0c6d3268cc4de15130300e72"
server_key_version: "encryptionkey@0"
I20260811 09:14:34.313421 22048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "bf26b3af800a44058cdc89076eab1a2e"
format_stamp: "Formatted at 2026-08-11 09:14:34 on dist-test-slave-ncp9"
server_key: "f5ccb7e44105fd771797fa060c095930"
server_key_iv: "5cd6618d0c6d3268cc4de15130300e72"
server_key_version: "encryptionkey@0"
I20260811 09:14:34.319027 22048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:14:34.322575 22084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:34.323956 22048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:14:34.324122 22048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "bf26b3af800a44058cdc89076eab1a2e"
format_stamp: "Formatted at 2026-08-11 09:14:34 on dist-test-slave-ncp9"
server_key: "f5ccb7e44105fd771797fa060c095930"
server_key_iv: "5cd6618d0c6d3268cc4de15130300e72"
server_key_version: "encryptionkey@0"
I20260811 09:14:34.324316 22048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:34.341845 22048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:34.345755 22048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:34.346027 22048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:34.347004 22048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:34.348368 22048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:34.348449 22048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:34.348599 22048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:34.348654 22048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:34.381049 22048 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:33635
I20260811 09:14:34.381093 22289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:33635 every 8 connection(s)
I20260811 09:14:34.382495 22048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:14:34.391158 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 22048
I20260811 09:14:34.391335 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:14:34.391754 17473 external_mini_cluster.cc:1428] Setting key dfe69dce6b2fd75d3dbdd02c2623731a
Aug 11 09:14:34 dist-test-slave-ncp9 krb5kdc[21595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439674, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:34.403939 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:34.385071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:58089 (local address 127.17.16.126:41017)
0811 09:14:34.385545 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:34.385550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:34.386473 (+ 923us) server_negotiation.cc:407] Connection header received
0811 09:14:34.388379 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:34.388383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:34.388453 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:34.388539 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:34.390027 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:34.390810 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:34.392567 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:34.393239 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:34.393366 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:34.397195 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:34.397202 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:34.397206 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:34.397245 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:34.400358 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:34.401306 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:34.401312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:34.401315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:34.401480 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:34.402013 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:34.402017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:34.402019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:34.402342 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:34.402483 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:34.403286 (+ 803us) server_negotiation.cc:299] Negotiation successful
0811 09:14:34.403546 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":227,"threads_started":1}
I20260811 09:14:34.404991 22297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:34.385560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41017 (local address 127.17.16.66:58089)
0811 09:14:34.386296 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:34.386362 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:34.388124 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:34.388736 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:34.388743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:34.389320 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:34.389875 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:34.389893 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:34.390986 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:34.390990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:34.392403 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:34.392417 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:34.393454 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:34.393460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:34.393737 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:34.394389 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:34.394417 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:34.396911 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:34.400556 (+ 3645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:34.400564 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:34.400578 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:34.401156 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:34.401628 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:34.401633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:34.401635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:34.401848 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:34.402521 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:34.402530 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:34.402871 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 09:14:34.403131 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 09:14:34.403729 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":296,"threads_started":1}
I20260811 09:14:34.406476 22290 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41017
I20260811 09:14:34.406785 22290 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:34.407557 22290 heartbeater.cc:511] Master 127.17.16.126:41017 requested a full tablet report, sending...
I20260811 09:14:34.409229 21668 ts_manager.cc:194] Registered new tserver with Master: bf26b3af800a44058cdc89076eab1a2e (127.17.16.66:33635)
I20260811 09:14:34.410377 21668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:58089
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
I20260811 09:14:34.894944 22036 heartbeater.cc:503] Master 127.17.16.126:41017 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
W20260811 09:14:35.128765 17473 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.17.16.67: real 0.734s user 0.000s sys 0.015s
I20260811 09:14:35.129637 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41017
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:14:35.412873 22290 heartbeater.cc:503] Master 127.17.16.126:41017 was elected leader, sending a full tablet report...
W20260811 09:14:35.491205 22300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:35.491496 22300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:35.491617 22300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:35.498869 22300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:35.498996 22300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:35.499156 22300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:14:35.509126 22300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:35.510473 22300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:35.511848 22300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:35.525014 22312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:35.525038 22315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:35.525504 22311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:35.525894 22300 server_base.cc:1034] running on GCE node
I20260811 09:14:35.526486 22300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:35.527408 22300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:35.528613 22300 hybrid_clock.cc:648] HybridClock initialized: now 1786439675528574 us; error 26 us; skew 500 ppm
I20260811 09:14:35.528940 22300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:35 dist-test-slave-ncp9 krb5kdc[21595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439675, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:35.532029 22300 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:14:35.533584 22300 webserver.cc:460] Webserver started at http://127.17.16.67:41185/ using document root <none> and password file <none>
I20260811 09:14:35.534255 22300 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:35.534379 22300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:35.534684 22300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:35.537618 22300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "cabed5f237ce4bc1a67d43dc757d7bd8"
format_stamp: "Formatted at 2026-08-11 09:14:35 on dist-test-slave-ncp9"
server_key: "9f4f28fc1dfe63d107797f96a34a6f3f"
server_key_iv: "628bc944440defc0250fe2282faeee40"
server_key_version: "encryptionkey@0"
I20260811 09:14:35.538386 22300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "cabed5f237ce4bc1a67d43dc757d7bd8"
format_stamp: "Formatted at 2026-08-11 09:14:35 on dist-test-slave-ncp9"
server_key: "9f4f28fc1dfe63d107797f96a34a6f3f"
server_key_iv: "628bc944440defc0250fe2282faeee40"
server_key_version: "encryptionkey@0"
I20260811 09:14:35.543501 22300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:14:35.546824 22329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:35.548032 22300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:14:35.548178 22300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "cabed5f237ce4bc1a67d43dc757d7bd8"
format_stamp: "Formatted at 2026-08-11 09:14:35 on dist-test-slave-ncp9"
server_key: "9f4f28fc1dfe63d107797f96a34a6f3f"
server_key_iv: "628bc944440defc0250fe2282faeee40"
server_key_version: "encryptionkey@0"
I20260811 09:14:35.548363 22300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:35.565501 22300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:35.569859 22300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:35.570139 22300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:35.571098 22300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:35.572420 22300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:35.572513 22300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:35.572619 22300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:35.572678 22300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:35.609680 22300 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:44271
I20260811 09:14:35.609721 22539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:44271 every 8 connection(s)
I20260811 09:14:35.611037 22300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:14:35.613873 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 22300
I20260811 09:14:35.614066 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:14:35.614531 17473 external_mini_cluster.cc:1428] Setting key b56502d637d449fb2d5355bc89604515
Aug 11 09:14:35 dist-test-slave-ncp9 krb5kdc[21595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439675, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:35.631313 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:35.613561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:36865 (local address 127.17.16.126:41017)
0811 09:14:35.614110 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:35.614116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:35.615213 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 09:14:35.617308 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:35.617314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:35.617396 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:35.617490 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:35.618993 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:35.619758 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:35.621343 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:35.621743 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:35.621854 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:35.625272 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:35.625277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:35.625281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:35.625313 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:35.627761 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:35.628810 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:35.628815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:35.628819 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:35.629007 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:35.629496 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:35.629500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:35.629502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:35.629791 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:35.629966 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:35.630670 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 09:14:35.630915 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":222,"threads_started":1}
I20260811 09:14:35.632242 22544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:35.614205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41017 (local address 127.17.16.67:36865)
0811 09:14:35.615028 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:35.615074 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:35.617035 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:35.617759 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:35.617767 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:35.618276 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:35.618819 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:35.618836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:35.619923 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:35.619927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:35.621202 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:35.621215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:35.621969 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:35.621977 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:35.622227 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:35.622817 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:35.622839 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:35.625081 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:35.628146 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:35.628155 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:35.628171 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:35.628714 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:35.629109 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:35.629112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:35.629115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:35.629370 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:35.629941 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:35.629950 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:35.630292 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 09:14:35.630537 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 09:14:35.631064 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":338,"threads_started":1}
I20260811 09:14:35.633657 22540 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41017
I20260811 09:14:35.633955 22540 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:35.634727 22540 heartbeater.cc:511] Master 127.17.16.126:41017 requested a full tablet report, sending...
I20260811 09:14:35.636286 21668 ts_manager.cc:194] Registered new tserver with Master: cabed5f237ce4bc1a67d43dc757d7bd8 (127.17.16.67:44271)
I20260811 09:14:35.637193 21668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:36865
I20260811 09:14:35.642285 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 09:14:35.679386 22543 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:14:35.676995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45774 (local address 127.17.16.126:41017)
0811 09:14:35.677529 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:35.677534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:35.677556 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:14:35.677716 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:35.677720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:35.677813 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:35.677900 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:35.679128 (+ 1228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45774: BlockingRecv error: recv got EOF from 127.0.0.1:45774 (error 108)
Metrics: {"server-negotiator.queue_time_us":410}
I20260811 09:14:35.682420 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21788
W20260811 09:14:35.685199 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21788: Not authorized: Client connection negotiation failed: client connection to 127.17.16.65:43683: server requires authentication, but client does not have Kerberos credentials available
I20260811 09:14:35.685410 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 21788
I20260811 09:14:35.708796 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22048
W20260811 09:14:35.711982 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22048: Not authorized: Client connection negotiation failed: client connection to 127.17.16.66:33635: server requires authentication, but client does not have Kerberos credentials available
I20260811 09:14:35.712124 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 22048
I20260811 09:14:35.739230 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22300
W20260811 09:14:35.742359 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22300: Not authorized: Client connection negotiation failed: client connection to 127.17.16.67:44271: server requires authentication, but client does not have Kerberos credentials available
I20260811 09:14:35.742532 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 22300
I20260811 09:14:35.789549 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21613
I20260811 09:14:35.790812 21731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:35.896168 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 21613
2026-08-11T09:14:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4502 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:14:36 dist-test-slave-ncp9 krb5kdc[22569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:14:36 dist-test-slave-ncp9 krb5kdc[22569](info): set up 2 sockets
Aug 11 09:14:36 dist-test-slave-ncp9 krb5kdc[22569](info): commencing operation
krb5kdc: starting...
W20260811 09:14:37.214956 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:14:37 dist-test-slave-ncp9 krb5kdc[22569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:14:37.424450 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:45147
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:44909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:45147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:37.745260 22588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:37.745546 22588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:37.745635 22588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:37.752766 22588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:37.752887 22588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:37.752966 22588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:37.753039 22588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:37.753096 22588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:37.762933 22588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:44909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:45147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:45147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:37.764264 22588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:37.765528 22588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:37.776914 22596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:37.777016 22595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:37.777053 22598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:37.777649 22588 server_base.cc:1034] running on GCE node
I20260811 09:14:37.778422 22588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:37.779753 22588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:37.780987 22588 hybrid_clock.cc:648] HybridClock initialized: now 1786439677780935 us; error 43 us; skew 500 ppm
I20260811 09:14:37.781302 22588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:37 dist-test-slave-ncp9 krb5kdc[22569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439677, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:37.787458 22588 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:14:37.789003 22588 webserver.cc:460] Webserver started at http://127.17.16.126:38073/ using document root <none> and password file <none>
I20260811 09:14:37.789635 22588 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:37.789749 22588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:37.790017 22588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:37.793139 22588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "01e322e6244e4fad9870225fdb0b523b"
format_stamp: "Formatted at 2026-08-11 09:14:37 on dist-test-slave-ncp9"
server_key: "b4860ea437f4f30e53a845ff20ebf96e"
server_key_iv: "895f340d99c751a8dab030f808fdcb78"
server_key_version: "encryptionkey@0"
I20260811 09:14:37.793931 22588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "01e322e6244e4fad9870225fdb0b523b"
format_stamp: "Formatted at 2026-08-11 09:14:37 on dist-test-slave-ncp9"
server_key: "b4860ea437f4f30e53a845ff20ebf96e"
server_key_iv: "895f340d99c751a8dab030f808fdcb78"
server_key_version: "encryptionkey@0"
I20260811 09:14:37.799221 22588 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:14:37.802686 22618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:37.803992 22588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:14:37.804159 22588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "01e322e6244e4fad9870225fdb0b523b"
format_stamp: "Formatted at 2026-08-11 09:14:37 on dist-test-slave-ncp9"
server_key: "b4860ea437f4f30e53a845ff20ebf96e"
server_key_iv: "895f340d99c751a8dab030f808fdcb78"
server_key_version: "encryptionkey@0"
I20260811 09:14:37.804349 22588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:37.824414 22588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:37.828540 22588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:37.828843 22588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:37.846913 22588 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:45147
I20260811 09:14:37.846944 22705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:45147 every 8 connection(s)
I20260811 09:14:37.848270 22588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:37.851979 22707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:37.854916 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 22588
I20260811 09:14:37.855182 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:37.855880 17473 external_mini_cluster.cc:1428] Setting key 9eac248e1dded92479826fd50ac1d344
I20260811 09:14:37.864800 22707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b: Bootstrap starting.
I20260811 09:14:37.868494 22707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:37.869822 22707 log.cc:824] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:14:37 dist-test-slave-ncp9 krb5kdc[22569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:37.872974 22707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b: No bootstrap required, opened a new log
I20260811 09:14:37.879593 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:37.857862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54656 (local address 127.17.16.126:45147)
0811 09:14:37.858716 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:37.858727 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:37.858772 (+ 45us) server_negotiation.cc:407] Connection header received
0811 09:14:37.860514 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:37.860542 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:37.861021 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:37.861382 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:37.863387 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:37.864974 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:37.866756 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:37.867484 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:37.867634 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:37.872338 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:37.872345 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:37.872360 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:37.872395 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:37.875423 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:37.876254 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:37.876261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:37.876267 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:37.876438 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:37.876859 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:37.876862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:37.876864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:37.877454 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:37.877692 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:37.878149 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 09:14:37.878513 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":380,"threads_started":1}
I20260811 09:14:37.881599 22707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e322e6244e4fad9870225fdb0b523b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45147 } }
I20260811 09:14:37.881954 22707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:37.882061 22707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e322e6244e4fad9870225fdb0b523b, State: Initialized, Role: FOLLOWER
I20260811 09:14:37.882679 22707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e322e6244e4fad9870225fdb0b523b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45147 } }
I20260811 09:14:37.882969 22707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:37.883119 22707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:37.883303 22707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:37.883431 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:14:37.885514 22707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e322e6244e4fad9870225fdb0b523b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45147 } }
I20260811 09:14:37.885983 22707 leader_election.cc:302] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01e322e6244e4fad9870225fdb0b523b; no voters:
I20260811 09:14:37.886552 22707 leader_election.cc:288] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:37.886694 22724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:37.887290 22724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [term 1 LEADER]: Becoming Leader. State: Replica: 01e322e6244e4fad9870225fdb0b523b, State: Running, Role: LEADER
I20260811 09:14:37.887864 22724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e322e6244e4fad9870225fdb0b523b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45147 } }
I20260811 09:14:37.890374 22707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:37.898017 22727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01e322e6244e4fad9870225fdb0b523b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e322e6244e4fad9870225fdb0b523b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45147 } } }
I20260811 09:14:37.898370 22727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:37.898893 22725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e322e6244e4fad9870225fdb0b523b. Latest consensus state: current_term: 1 leader_uuid: "01e322e6244e4fad9870225fdb0b523b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e322e6244e4fad9870225fdb0b523b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45147 } } }
I20260811 09:14:37.899170 22725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:37.905208 22750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:37.910403 22750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:37.919178 22750 catalog_manager.cc:1347] Generated new cluster ID: a3bf2d12aa364de68ac49d4d8ab7e09b
I20260811 09:14:37.919291 22750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:37.936872 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:37.929341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.17.16.126:45147)
0811 09:14:37.929559 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:37.929566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:37.930136 (+ 570us) server_negotiation.cc:407] Connection header received
0811 09:14:37.930316 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:37.930320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:37.930388 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:37.930479 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:37.931533 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:37.932298 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:37.933635 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:37.934048 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:37.934182 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:37.935248 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:37.935253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:37.935257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:37.936064 (+ 807us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:37.936143 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:37.936170 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:37.936265 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:37.936425 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 09:14:37.936525 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:14:37.940537 22750 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:37.941941 22750 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:37.948407 22750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01e322e6244e4fad9870225fdb0b523b: Generated new TSK 0
I20260811 09:14:37.949465 22750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:14:37.959641 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22588
I20260811 09:14:37.960969 22701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:38.052382 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 22588
2026-08-11T09:14:38Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2152 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:14:38 dist-test-slave-ncp9 krb5kdc[22769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:14:38 dist-test-slave-ncp9 krb5kdc[22769](info): set up 2 sockets
Aug 11 09:14:38 dist-test-slave-ncp9 krb5kdc[22769](info): commencing operation
krb5kdc: starting...
W20260811 09:14:39.261763 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:14:39 dist-test-slave-ncp9 krb5kdc[22769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:14:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:14:39.484676 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:44631
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:44631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:39.812453 22792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:39.812758 22792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:39.812880 22792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:39.819972 22792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:39.820082 22792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:39.820158 22792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:39.820232 22792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:39.820294 22792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:39.829859 22792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:44631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:44631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:39.831138 22792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:39.832399 22792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:39.843489 22799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:39.843559 22800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:39.843709 22802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:39.844255 22792 server_base.cc:1034] running on GCE node
I20260811 09:14:39.844964 22792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:39.846258 22792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:39.847477 22792 hybrid_clock.cc:648] HybridClock initialized: now 1786439679847415 us; error 32 us; skew 500 ppm
I20260811 09:14:39.847801 22792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:39 dist-test-slave-ncp9 krb5kdc[22769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439679, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:39.850798 22792 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:14:39.852206 22792 webserver.cc:460] Webserver started at http://127.17.16.126:35111/ using document root <none> and password file <none>
I20260811 09:14:39.852843 22792 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:39.852996 22792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:39.853291 22792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:39.856436 22792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "24df2be2826f4f278ae01e24092d7c71"
format_stamp: "Formatted at 2026-08-11 09:14:39 on dist-test-slave-ncp9"
server_key: "5cb415f3f438731528ad18be1c2e311f"
server_key_iv: "f26960f68b016744bc4770eb436358ad"
server_key_version: "encryptionkey@0"
I20260811 09:14:39.857231 22792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "24df2be2826f4f278ae01e24092d7c71"
format_stamp: "Formatted at 2026-08-11 09:14:39 on dist-test-slave-ncp9"
server_key: "5cb415f3f438731528ad18be1c2e311f"
server_key_iv: "f26960f68b016744bc4770eb436358ad"
server_key_version: "encryptionkey@0"
I20260811 09:14:39.862114 22792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 09:14:39.865492 22812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:39.866767 22792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:14:39.866930 22792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "24df2be2826f4f278ae01e24092d7c71"
format_stamp: "Formatted at 2026-08-11 09:14:39 on dist-test-slave-ncp9"
server_key: "5cb415f3f438731528ad18be1c2e311f"
server_key_iv: "f26960f68b016744bc4770eb436358ad"
server_key_version: "encryptionkey@0"
I20260811 09:14:39.867120 22792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:39.880775 22792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:39.884604 22792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:39.884876 22792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:39.902081 22792 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:44631
I20260811 09:14:39.902103 22899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:44631 every 8 connection(s)
I20260811 09:14:39.903403 22792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:39.906713 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 22792
I20260811 09:14:39.906919 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:39.906952 22907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:39.907565 17473 external_mini_cluster.cc:1428] Setting key 769e3fd9de12593f0287329436041b35
I20260811 09:14:39.920874 22907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71: Bootstrap starting.
Aug 11 09:14:39 dist-test-slave-ncp9 krb5kdc[22769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:39.923974 22907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:39.925351 22907 log.cc:824] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:39.928658 22907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71: No bootstrap required, opened a new log
I20260811 09:14:39.932093 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:39.909593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44254 (local address 127.17.16.126:44631)
0811 09:14:39.910412 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:39.910427 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:39.910480 (+ 53us) server_negotiation.cc:407] Connection header received
0811 09:14:39.913324 (+ 2844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:39.913354 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:39.913861 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:39.914293 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:39.915593 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:39.917565 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:39.918866 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:39.919895 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:39.920090 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:39.923919 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:39.923931 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:39.923952 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:39.924008 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:39.927557 (+ 3549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:39.928224 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:39.928233 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:39.928240 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:39.928439 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:39.928797 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:39.928802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:39.928804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:39.929538 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:39.929856 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:39.930260 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 09:14:39.930722 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":377,"threads_started":1}
I20260811 09:14:39.938004 22907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24df2be2826f4f278ae01e24092d7c71" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44631 } }
I20260811 09:14:39.938354 22907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:39.938426 22907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24df2be2826f4f278ae01e24092d7c71, State: Initialized, Role: FOLLOWER
I20260811 09:14:39.938951 22907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24df2be2826f4f278ae01e24092d7c71" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44631 } }
I20260811 09:14:39.939162 22907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:39.939258 22907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:39.939376 22907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:39.941629 22907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24df2be2826f4f278ae01e24092d7c71" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44631 } }
I20260811 09:14:39.942068 22907 leader_election.cc:302] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24df2be2826f4f278ae01e24092d7c71; no voters:
I20260811 09:14:39.942603 22907 leader_election.cc:288] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:39.942720 22918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:39.943361 22918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [term 1 LEADER]: Becoming Leader. State: Replica: 24df2be2826f4f278ae01e24092d7c71, State: Running, Role: LEADER
I20260811 09:14:39.943905 22918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24df2be2826f4f278ae01e24092d7c71" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44631 } }
I20260811 09:14:39.944545 22907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:39.946496 22921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24df2be2826f4f278ae01e24092d7c71. Latest consensus state: current_term: 1 leader_uuid: "24df2be2826f4f278ae01e24092d7c71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24df2be2826f4f278ae01e24092d7c71" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44631 } } }
I20260811 09:14:39.946506 22920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24df2be2826f4f278ae01e24092d7c71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24df2be2826f4f278ae01e24092d7c71" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44631 } } }
I20260811 09:14:39.946933 22920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:39.946928 22921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:39.949398 22926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:39.953537 22926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:39.961977 22926 catalog_manager.cc:1347] Generated new cluster ID: 99945089074d45d5b7d66334056e40ee
I20260811 09:14:39.962098 22926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:39.984511 22926 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:39.985980 22926 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:40.005347 22926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24df2be2826f4f278ae01e24092d7c71: Generated new TSK 0
I20260811 09:14:40.006255 22926 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:14:40.064942 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44631
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:40.388451 22956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:40.388739 22956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:40.388880 22956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:40.395875 22956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:40.396009 22956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:40.396174 22956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:14:40.405985 22956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:40.407238 22956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:40.408515 22956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:40.421177 22965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:40.421259 22964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:40.421155 22971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:40.422012 22956 server_base.cc:1034] running on GCE node
I20260811 09:14:40.422582 22956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:40.423491 22956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:40.424690 22956 hybrid_clock.cc:648] HybridClock initialized: now 1786439680424639 us; error 44 us; skew 500 ppm
I20260811 09:14:40.425034 22956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:40 dist-test-slave-ncp9 krb5kdc[22769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439680, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:40.428104 22956 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:14:40.429533 22956 webserver.cc:460] Webserver started at http://127.17.16.65:43827/ using document root <none> and password file <none>
I20260811 09:14:40.430197 22956 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:40.430307 22956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:40.430603 22956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:40.433501 22956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "21cc565f28224d5782812653e846a948"
format_stamp: "Formatted at 2026-08-11 09:14:40 on dist-test-slave-ncp9"
server_key: "79bcd779f749f315c5a475ffc29362d7"
server_key_iv: "746b4dd997085db848a0049564910b96"
server_key_version: "encryptionkey@0"
I20260811 09:14:40.434304 22956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "21cc565f28224d5782812653e846a948"
format_stamp: "Formatted at 2026-08-11 09:14:40 on dist-test-slave-ncp9"
server_key: "79bcd779f749f315c5a475ffc29362d7"
server_key_iv: "746b4dd997085db848a0049564910b96"
server_key_version: "encryptionkey@0"
I20260811 09:14:40.439674 22956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:14:40.443217 22978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:40.444552 22956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 09:14:40.444708 22956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "21cc565f28224d5782812653e846a948"
format_stamp: "Formatted at 2026-08-11 09:14:40 on dist-test-slave-ncp9"
server_key: "79bcd779f749f315c5a475ffc29362d7"
server_key_iv: "746b4dd997085db848a0049564910b96"
server_key_version: "encryptionkey@0"
I20260811 09:14:40.444916 22956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:40.457077 22956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:40.460757 22956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:40.461063 22956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:40.461969 22956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:40.463286 22956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:40.463366 22956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:40.463478 22956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:40.463531 22956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:40.494904 22956 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:39793
I20260811 09:14:40.494951 23188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:39793 every 8 connection(s)
I20260811 09:14:40.496210 22956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:14:40.505604 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 22956
I20260811 09:14:40.505829 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:14:40.506453 17473 external_mini_cluster.cc:1428] Setting key 5396fd53dd63d93fef8e5fd5e8b948fd
Aug 11 09:14:40 dist-test-slave-ncp9 krb5kdc[22769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439680, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:40.515786 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:40.498795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:44379 (local address 127.17.16.126:44631)
0811 09:14:40.499245 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:40.499250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:40.500145 (+ 895us) server_negotiation.cc:407] Connection header received
0811 09:14:40.502007 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:40.502012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:40.502108 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:40.502224 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:40.503691 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:40.504450 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:40.506255 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:40.506691 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:40.506804 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:40.510315 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:40.510320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:40.510323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:40.510354 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:40.512741 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:40.513484 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:40.513488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:40.513491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:40.513613 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:40.514055 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:40.514059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:40.514061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:40.514339 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:40.514472 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:40.515196 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 09:14:40.515424 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":224,"threads_started":1}
I20260811 09:14:40.516789 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:40.499278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:44631 (local address 127.17.16.65:44379)
0811 09:14:40.499980 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:40.500026 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:40.501742 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:40.502467 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:40.502474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:40.502967 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:40.503506 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:40.503533 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:40.504610 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:40.504613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:40.506110 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:40.506124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:40.506914 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:40.506922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:40.507151 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:40.507868 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:40.507891 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:40.510098 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:40.512885 (+ 2787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:40.512915 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:40.512932 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:40.513352 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:40.513726 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:40.513731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:40.513735 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:40.513923 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:40.514478 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:40.514486 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:40.514824 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 09:14:40.515061 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 09:14:40.515621 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":280,"threads_started":1}
I20260811 09:14:40.518538 23189 heartbeater.cc:347] Connected to a master server at 127.17.16.126:44631
I20260811 09:14:40.518847 23189 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:40.519601 23189 heartbeater.cc:511] Master 127.17.16.126:44631 requested a full tablet report, sending...
I20260811 09:14:40.521965 22844 ts_manager.cc:194] Registered new tserver with Master: 21cc565f28224d5782812653e846a948 (127.17.16.65:39793)
I20260811 09:14:40.523967 22844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:44379
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:14:40.560714 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44631
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:40.884606 23198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:40.884935 23198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:40.885121 23198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:40.892515 23198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:40.892650 23198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:40.892830 23198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:14:40.902841 23198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:40.904129 23198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:40.905479 23198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:40.918996 23211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:40.919028 23207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:40.919335 23206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:40.919991 23198 server_base.cc:1034] running on GCE node
I20260811 09:14:40.920547 23198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:40.921483 23198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:40.922685 23198 hybrid_clock.cc:648] HybridClock initialized: now 1786439680922634 us; error 45 us; skew 500 ppm
I20260811 09:14:40.922981 23198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:40 dist-test-slave-ncp9 krb5kdc[22769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439680, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:40.926086 23198 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:14:40.927443 23198 webserver.cc:460] Webserver started at http://127.17.16.66:39695/ using document root <none> and password file <none>
I20260811 09:14:40.928076 23198 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:40.928206 23198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:40.928495 23198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:40.931449 23198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "d3f28cdf46634eb7b528b0ea946117f9"
format_stamp: "Formatted at 2026-08-11 09:14:40 on dist-test-slave-ncp9"
server_key: "f2d5801ea962424ffcbbb3f55c819700"
server_key_iv: "e47ac84953c938727d35f1ad3e03c2a2"
server_key_version: "encryptionkey@0"
I20260811 09:14:40.932200 23198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "d3f28cdf46634eb7b528b0ea946117f9"
format_stamp: "Formatted at 2026-08-11 09:14:40 on dist-test-slave-ncp9"
server_key: "f2d5801ea962424ffcbbb3f55c819700"
server_key_iv: "e47ac84953c938727d35f1ad3e03c2a2"
server_key_version: "encryptionkey@0"
I20260811 09:14:40.937191 23198 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 09:14:40.940426 23225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:40.941761 23198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:14:40.941915 23198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "d3f28cdf46634eb7b528b0ea946117f9"
format_stamp: "Formatted at 2026-08-11 09:14:40 on dist-test-slave-ncp9"
server_key: "f2d5801ea962424ffcbbb3f55c819700"
server_key_iv: "e47ac84953c938727d35f1ad3e03c2a2"
server_key_version: "encryptionkey@0"
I20260811 09:14:40.942095 23198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:40.965463 23198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:40.969652 23198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:40.969956 23198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:40.970952 23198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:40.972252 23198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:40.972325 23198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:40.972415 23198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:40.972457 23198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:41.004784 23198 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:41025
I20260811 09:14:41.004858 23414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:41025 every 8 connection(s)
I20260811 09:14:41.006119 23198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:14:41.011919 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 23198
I20260811 09:14:41.012118 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:14:41.012530 17473 external_mini_cluster.cc:1428] Setting key d8ffaa3483486865d69199df76abbd2a
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[22769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439680, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:41.025084 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:41.008576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:33201 (local address 127.17.16.126:44631)
0811 09:14:41.008768 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:41.008773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:41.009852 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 09:14:41.011657 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:41.011664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:41.011744 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:41.011833 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:41.013422 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.014143 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.015665 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.016074 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.016185 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:41.019546 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:41.019552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:41.019555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:41.019587 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:41.022089 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.022809 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.022813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.022815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.022944 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.023378 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.023382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.023384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.023653 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:41.023798 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:41.024480 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 09:14:41.024736 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:14:41.026018 23417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:41.009001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:44631 (local address 127.17.16.66:33201)
0811 09:14:41.009695 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:41.009733 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:41.011367 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:41.012100 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:41.012111 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:41.012634 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:41.013248 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:41.013265 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.014304 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.014308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:41.015511 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:41.015523 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.016306 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.016314 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:41.016532 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:41.017206 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:41.017228 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:41.019328 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:41.022246 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:41.022255 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:41.022272 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:41.022677 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:41.023057 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:41.023062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:41.023065 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:41.023258 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:41.023790 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:41.023798 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:41.024111 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 09:14:41.024351 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 09:14:41.024868 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":257,"threads_started":1}
I20260811 09:14:41.027416 23415 heartbeater.cc:347] Connected to a master server at 127.17.16.126:44631
I20260811 09:14:41.027738 23415 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:41.028612 23415 heartbeater.cc:511] Master 127.17.16.126:44631 requested a full tablet report, sending...
I20260811 09:14:41.030252 22844 ts_manager.cc:194] Registered new tserver with Master: d3f28cdf46634eb7b528b0ea946117f9 (127.17.16.66:41025)
I20260811 09:14:41.031373 22844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:33201
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:14:41.061839 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44631
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:41.384567 23424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:41.384845 23424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:41.385006 23424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:41.392115 23424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:41.392217 23424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:41.392335 23424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:14:41.401830 23424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:41.403095 23424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:41.404304 23424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:41.416687 23434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:41.416640 23435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:41.416607 23439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:41.417724 23424 server_base.cc:1034] running on GCE node
I20260811 09:14:41.418246 23424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:41.419094 23424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:41.420291 23424 hybrid_clock.cc:648] HybridClock initialized: now 1786439681420245 us; error 38 us; skew 500 ppm
I20260811 09:14:41.420589 23424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[22769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439681, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:41.423619 23424 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:14:41.425113 23424 webserver.cc:460] Webserver started at http://127.17.16.67:36007/ using document root <none> and password file <none>
I20260811 09:14:41.425760 23424 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:41.425879 23424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:41.426172 23424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:41.429016 23424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "f69d9561adab491f9b619e5ba5556af9"
format_stamp: "Formatted at 2026-08-11 09:14:41 on dist-test-slave-ncp9"
server_key: "9a1832dc233d73cacbf1d8e0c5c43f38"
server_key_iv: "3af1d9031654d251e1806aadf8f32a40"
server_key_version: "encryptionkey@0"
I20260811 09:14:41.429777 23424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "f69d9561adab491f9b619e5ba5556af9"
format_stamp: "Formatted at 2026-08-11 09:14:41 on dist-test-slave-ncp9"
server_key: "9a1832dc233d73cacbf1d8e0c5c43f38"
server_key_iv: "3af1d9031654d251e1806aadf8f32a40"
server_key_version: "encryptionkey@0"
I20260811 09:14:41.434702 23424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 09:14:41.438035 23455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:41.439275 23424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:14:41.439424 23424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "f69d9561adab491f9b619e5ba5556af9"
format_stamp: "Formatted at 2026-08-11 09:14:41 on dist-test-slave-ncp9"
server_key: "9a1832dc233d73cacbf1d8e0c5c43f38"
server_key_iv: "3af1d9031654d251e1806aadf8f32a40"
server_key_version: "encryptionkey@0"
I20260811 09:14:41.439596 23424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:41.461251 23424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:41.464784 23424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:41.465072 23424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:41.465963 23424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:41.467216 23424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:41.467293 23424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:41.467414 23424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:41.467468 23424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:41.498418 23424 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:38951
I20260811 09:14:41.498448 23641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:38951 every 8 connection(s)
I20260811 09:14:41.499922 23424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:14:41.501103 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 23424
I20260811 09:14:41.501314 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:14:41.501837 17473 external_mini_cluster.cc:1428] Setting key b03218f6091759e0e1dbf2caefee1512
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[22769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439681, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:41.519461 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:41.502846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:59335 (local address 127.17.16.126:44631)
0811 09:14:41.503044 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:41.503050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:41.504366 (+ 1316us) server_negotiation.cc:407] Connection header received
0811 09:14:41.506329 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:41.506333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:41.506400 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:41.506477 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:41.507872 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.508623 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.510126 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.510576 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.510670 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:41.513934 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:41.513940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:41.513943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:41.513976 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:41.516641 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.517452 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.517456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.517459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.517600 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.517967 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.517970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.517971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.518263 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:41.518374 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:41.519034 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 09:14:41.519212 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 09:14:41.520462 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:41.503345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:44631 (local address 127.17.16.67:59335)
0811 09:14:41.504195 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:41.504244 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:41.506096 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:41.506678 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:41.506686 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:41.507193 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:41.507733 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:41.507750 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.508737 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.508741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:41.510000 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:41.510012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.510776 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.510780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:41.510974 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:41.511573 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:41.511600 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:41.513717 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:41.516788 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:41.516798 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:41.516816 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:41.517325 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:41.517687 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:41.517691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:41.517693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:41.517879 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:41.518364 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:41.518370 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:41.518703 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 09:14:41.518943 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 09:14:41.519360 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":326,"threads_started":1}
I20260811 09:14:41.521883 23642 heartbeater.cc:347] Connected to a master server at 127.17.16.126:44631
I20260811 09:14:41.522158 23642 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:41.522871 23642 heartbeater.cc:511] Master 127.17.16.126:44631 requested a full tablet report, sending...
I20260811 09:14:41.524224 22844 ts_manager.cc:194] Registered new tserver with Master: f69d9561adab491f9b619e5ba5556af9 (127.17.16.67:38951)
I20260811 09:14:41.525048 22844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:59335
I20260811 09:14:41.527056 23189 heartbeater.cc:503] Master 127.17.16.126:44631 was elected leader, sending a full tablet report...
I20260811 09:14:41.529661 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[22769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439681, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[22769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439681, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:41.580646 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:41.566946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44262 (local address 127.17.16.126:44631)
0811 09:14:41.567119 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:41.567124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:41.567544 (+ 420us) server_negotiation.cc:407] Connection header received
0811 09:14:41.567700 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:41.567704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:41.567768 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:41.567856 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:41.569205 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.569930 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.571246 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.571741 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.571840 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:41.575528 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:41.575533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:41.575536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:41.575568 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:41.578083 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.578935 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.578940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.578942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.579079 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.579479 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.579482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.579484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.579794 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:41.579973 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:41.580224 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 09:14:41.580384 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 09:14:41.581359 22844 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44262
I20260811 09:14:41.583369 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22956
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[22769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439681, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:14:41.598973 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:41.584314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.17.16.65:39793)
0811 09:14:41.585079 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:41.585085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:41.585108 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:14:41.585168 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:41.585175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:41.585370 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:41.585493 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:41.586613 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.587627 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.588952 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.589515 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.589616 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:41.593495 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:41.593507 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:41.593521 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:41.593558 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:41.596444 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.597193 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.597201 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.597203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.597335 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.597675 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.597678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.597680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.598018 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:41.598239 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:41.598469 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 09:14:41.598685 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":437,"threads_started":1}
W20260811 09:14:41.600236 23181 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45466
W20260811 09:14:41.601372 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22956: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:14:41.601562 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 22956
I20260811 09:14:41.626777 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 23198
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[22769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439681, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:14:41.642614 23659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:41.627672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46922 (local address 127.17.16.66:41025)
0811 09:14:41.628187 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:41.628194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:41.628220 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:14:41.628280 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:41.628287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:41.628490 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:41.628619 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:41.629866 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.630861 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.632136 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.632678 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.632770 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:41.636960 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:41.636975 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:41.636998 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:41.637038 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:41.639866 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.640523 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.640530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.640533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.640671 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.641090 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.641094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.641096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.641443 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:41.641782 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:41.642020 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 09:14:41.642254 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":254,"threads_started":1}
W20260811 09:14:41.643885 23407 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46922
W20260811 09:14:41.644936 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 23198: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:14:41.645114 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 23198
I20260811 09:14:41.667754 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 23424
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[22769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439681, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:14:41.683607 23663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:41.668723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.17.16.67:38951)
0811 09:14:41.669485 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:41.669492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:41.669521 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:14:41.669599 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:41.669606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:41.669841 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:41.669967 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:41.671468 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.672557 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.673760 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:41.674310 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:41.674405 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:41.678551 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:41.678563 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:41.678578 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:41.678625 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:41.681289 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.681878 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.681885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.681888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.682009 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:41.682320 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:41.682323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:41.682325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:41.682673 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:41.682896 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:41.683107 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 09:14:41.683310 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":450,"threads_started":1}
W20260811 09:14:41.684962 23637 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41424
W20260811 09:14:41.686072 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 23424: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:14:41.686239 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 23424
I20260811 09:14:41.708711 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22792
I20260811 09:14:41.710011 22895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:41.824348 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 22792
2026-08-11T09:14:41Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3778 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[23669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[23669](info): set up 2 sockets
Aug 11 09:14:41 dist-test-slave-ncp9 krb5kdc[23669](info): commencing operation
krb5kdc: starting...
W20260811 09:14:43.060456 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:14:43 dist-test-slave-ncp9 krb5kdc[23669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:14:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:14:43.270254 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:33477
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:42375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:33477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:43.598012 23698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:43.598284 23698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:43.598423 23698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:43.605432 23698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:43.605536 23698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:43.605613 23698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:43.605683 23698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:43.605748 23698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:43.615125 23698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:33477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:33477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:43.616354 23698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:43.617766 23698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:43.628969 23712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:43.629010 23710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:43.629253 23709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:43.630235 23698 server_base.cc:1034] running on GCE node
I20260811 09:14:43.630945 23698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:43.632241 23698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:43.633445 23698 hybrid_clock.cc:648] HybridClock initialized: now 1786439683633391 us; error 36 us; skew 500 ppm
I20260811 09:14:43.633767 23698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:43 dist-test-slave-ncp9 krb5kdc[23669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439683, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:43.639410 23698 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:14:43.640817 23698 webserver.cc:460] Webserver started at http://127.17.16.126:41119/ using document root <none> and password file <none>
I20260811 09:14:43.641487 23698 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:43.641587 23698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:43.641847 23698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:43.644671 23698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "5645637927ff44df9c8c30bdd82f2f93"
format_stamp: "Formatted at 2026-08-11 09:14:43 on dist-test-slave-ncp9"
server_key: "08e4b6170acd271bbcf5abc8a40f999d"
server_key_iv: "1e58e5f5175fe806bf488d713a9beb1b"
server_key_version: "encryptionkey@0"
I20260811 09:14:43.645416 23698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "5645637927ff44df9c8c30bdd82f2f93"
format_stamp: "Formatted at 2026-08-11 09:14:43 on dist-test-slave-ncp9"
server_key: "08e4b6170acd271bbcf5abc8a40f999d"
server_key_iv: "1e58e5f5175fe806bf488d713a9beb1b"
server_key_version: "encryptionkey@0"
I20260811 09:14:43.650162 23698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 09:14:43.653296 23725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:43.654521 23698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:14:43.654661 23698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "5645637927ff44df9c8c30bdd82f2f93"
format_stamp: "Formatted at 2026-08-11 09:14:43 on dist-test-slave-ncp9"
server_key: "08e4b6170acd271bbcf5abc8a40f999d"
server_key_iv: "1e58e5f5175fe806bf488d713a9beb1b"
server_key_version: "encryptionkey@0"
I20260811 09:14:43.654834 23698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:43.666807 23698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:43.670408 23698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:43.670663 23698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:43.687912 23698 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:33477
I20260811 09:14:43.687918 23828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:33477 every 8 connection(s)
I20260811 09:14:43.689236 23698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:43.692768 23829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:43.699146 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 23698
I20260811 09:14:43.699359 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:43.700186 17473 external_mini_cluster.cc:1428] Setting key 22ce9c3d20e70d3196df81e28e25b3b7
I20260811 09:14:43.703845 23829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93: Bootstrap starting.
I20260811 09:14:43.707306 23829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:43.708632 23829 log.cc:824] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:43.711513 23829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93: No bootstrap required, opened a new log
Aug 11 09:14:43 dist-test-slave-ncp9 krb5kdc[23669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:43.721004 23829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5645637927ff44df9c8c30bdd82f2f93" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33477 } }
I20260811 09:14:43.721534 23829 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:43.721637 23829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5645637927ff44df9c8c30bdd82f2f93, State: Initialized, Role: FOLLOWER
I20260811 09:14:43.722244 23829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5645637927ff44df9c8c30bdd82f2f93" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33477 } }
I20260811 09:14:43.722501 23829 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:43.722647 23829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:43.722810 23829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:43.724668 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:43.702491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50696 (local address 127.17.16.126:33477)
0811 09:14:43.703359 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:43.703371 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:43.703413 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:14:43.705428 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:43.705461 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:43.705851 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:43.706190 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:43.708105 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:43.709851 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:43.711136 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:43.712149 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:43.712341 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:43.716159 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:43.716170 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:43.716190 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:43.716235 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:43.720342 (+ 4107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:43.721082 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:43.721092 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:43.721101 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:43.721309 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:43.721707 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:43.721713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:43.721717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:43.722410 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:43.722719 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:43.723162 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 09:14:43.723615 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":417,"threads_started":1}
I20260811 09:14:43.725425 23829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5645637927ff44df9c8c30bdd82f2f93" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33477 } }
I20260811 09:14:43.725920 23829 leader_election.cc:302] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5645637927ff44df9c8c30bdd82f2f93; no voters:
I20260811 09:14:43.726544 23829 leader_election.cc:288] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:43.726671 23838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:43.727632 23838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [term 1 LEADER]: Becoming Leader. State: Replica: 5645637927ff44df9c8c30bdd82f2f93, State: Running, Role: LEADER
I20260811 09:14:43.728252 23838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5645637927ff44df9c8c30bdd82f2f93" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33477 } }
I20260811 09:14:43.728829 23829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:43.731803 23844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5645637927ff44df9c8c30bdd82f2f93. Latest consensus state: current_term: 1 leader_uuid: "5645637927ff44df9c8c30bdd82f2f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5645637927ff44df9c8c30bdd82f2f93" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33477 } } }
I20260811 09:14:43.731808 23842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5645637927ff44df9c8c30bdd82f2f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5645637927ff44df9c8c30bdd82f2f93" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33477 } } }
I20260811 09:14:43.732911 23844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:43.733389 23842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:43.734550 23855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:43.739773 23855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:43.748975 23855 catalog_manager.cc:1347] Generated new cluster ID: 60a1bc316a1f4d97b443762fb7f653cb
I20260811 09:14:43.749095 23855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:43.776129 23855 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:43.777585 23855 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:43.784876 23855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5645637927ff44df9c8c30bdd82f2f93: Generated new TSK 0
I20260811 09:14:43.786072 23855 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:14:43.842428 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:33477
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:42375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:44.164271 23878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:44.164541 23878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:44.164665 23878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:44.171720 23878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:44.171818 23878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:44.171967 23878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:14:44.181583 23878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:44.182852 23878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:44.184100 23878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:44.196699 23886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:44.196792 23889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:44.196704 23887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:44.197889 23878 server_base.cc:1034] running on GCE node
I20260811 09:14:44.198505 23878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:44.199424 23878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:44.200644 23878 hybrid_clock.cc:648] HybridClock initialized: now 1786439684200585 us; error 55 us; skew 500 ppm
I20260811 09:14:44.200996 23878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:44 dist-test-slave-ncp9 krb5kdc[23669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439684, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:44.204006 23878 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:14:44.205406 23878 webserver.cc:460] Webserver started at http://127.17.16.65:37891/ using document root <none> and password file <none>
I20260811 09:14:44.206019 23878 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:44.206116 23878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:44.206374 23878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:44.209111 23878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "827c2d583b0048afa5f30a94dee2cd3f"
format_stamp: "Formatted at 2026-08-11 09:14:44 on dist-test-slave-ncp9"
server_key: "cadae088ca5d02d7c96088cb1c4cec82"
server_key_iv: "c2bdd16eb3069d346f1b400245b41ec8"
server_key_version: "encryptionkey@0"
I20260811 09:14:44.209841 23878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "827c2d583b0048afa5f30a94dee2cd3f"
format_stamp: "Formatted at 2026-08-11 09:14:44 on dist-test-slave-ncp9"
server_key: "cadae088ca5d02d7c96088cb1c4cec82"
server_key_iv: "c2bdd16eb3069d346f1b400245b41ec8"
server_key_version: "encryptionkey@0"
I20260811 09:14:44.214591 23878 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:14:44.217824 23901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:44.219050 23878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:14:44.219195 23878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "827c2d583b0048afa5f30a94dee2cd3f"
format_stamp: "Formatted at 2026-08-11 09:14:44 on dist-test-slave-ncp9"
server_key: "cadae088ca5d02d7c96088cb1c4cec82"
server_key_iv: "c2bdd16eb3069d346f1b400245b41ec8"
server_key_version: "encryptionkey@0"
I20260811 09:14:44.219380 23878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:44.251606 23878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:44.255336 23878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:44.255596 23878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:44.256528 23878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:44.257869 23878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:44.257941 23878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:44.258068 23878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:44.258169 23878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 09:14:44.288264 23878 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:32833
I20260811 09:14:44.288300 24094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:32833 every 8 connection(s)
I20260811 09:14:44.289546 23878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:14:44.292794 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 23878
I20260811 09:14:44.293030 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:14:44.293866 17473 external_mini_cluster.cc:1428] Setting key e0f0caa2e07728fde34aa2e13666c6a8
Aug 11 09:14:44 dist-test-slave-ncp9 krb5kdc[23669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439684, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:44.309906 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:44.291933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:44819 (local address 127.17.16.126:33477)
0811 09:14:44.292395 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:44.292401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:44.293542 (+ 1141us) server_negotiation.cc:407] Connection header received
0811 09:14:44.295781 (+ 2239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:44.295786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:44.295859 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:44.295946 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:44.297474 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:44.298504 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:44.300047 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:44.300569 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:44.300703 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:44.304295 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:44.304303 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:44.304307 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:44.304350 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:44.306794 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:44.307676 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:44.307679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:44.307681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:44.307790 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:44.308213 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:44.308216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:44.308218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:44.308457 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:44.308560 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:44.309475 (+ 915us) server_negotiation.cc:299] Negotiation successful
0811 09:14:44.309647 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":222,"threads_started":1}
I20260811 09:14:44.311241 24105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:44.292530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:33477 (local address 127.17.16.65:44819)
0811 09:14:44.293345 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:44.293405 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:44.295477 (+ 2072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:44.296181 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:44.296189 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:44.296696 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:44.297293 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:44.297311 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:44.298657 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:44.298660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:44.299901 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:44.299913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:44.300776 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:44.300780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:44.301030 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:44.301792 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:44.301833 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:44.304067 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:44.306936 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:44.306945 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:44.306961 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:44.307567 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:44.307887 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:44.307891 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:44.307895 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:44.308120 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:44.308582 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:44.308590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:44.309056 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 09:14:44.309389 (+ 333us) client_negotiation.cc:240] Negotiation successful
0811 09:14:44.309895 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":337,"threads_started":1}
I20260811 09:14:44.312763 24095 heartbeater.cc:347] Connected to a master server at 127.17.16.126:33477
I20260811 09:14:44.313083 24095 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:44.313868 24095 heartbeater.cc:511] Master 127.17.16.126:33477 requested a full tablet report, sending...
I20260811 09:14:44.317008 23750 ts_manager.cc:194] Registered new tserver with Master: 827c2d583b0048afa5f30a94dee2cd3f (127.17.16.65:32833)
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
I20260811 09:14:44.318506 23750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:44819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:14:44.346462 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:33477
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:42375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:44.670706 24110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:44.670958 24110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:44.671069 24110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:44.678112 24110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:44.678212 24110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:44.678335 24110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:14:44.687809 24110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:44.688990 24110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:44.690251 24110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:44.703294 24115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:44.703322 24116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:44.703255 24118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:44.705188 24110 server_base.cc:1034] running on GCE node
I20260811 09:14:44.705845 24110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:44.706816 24110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:44.708003 24110 hybrid_clock.cc:648] HybridClock initialized: now 1786439684707944 us; error 54 us; skew 500 ppm
I20260811 09:14:44.708307 24110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:44 dist-test-slave-ncp9 krb5kdc[23669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439684, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:44.711315 24110 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:14:44.712990 24110 webserver.cc:460] Webserver started at http://127.17.16.66:36941/ using document root <none> and password file <none>
I20260811 09:14:44.713742 24110 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:44.713882 24110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:44.714304 24110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:44.717689 24110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "1b172a701cc64558bf5d87a1cef6e8e0"
format_stamp: "Formatted at 2026-08-11 09:14:44 on dist-test-slave-ncp9"
server_key: "c4b56a2d27b48e60fe739bfd1d6e0ac4"
server_key_iv: "71c11f3670ac2133c590982e7aaf1b8c"
server_key_version: "encryptionkey@0"
I20260811 09:14:44.718570 24110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "1b172a701cc64558bf5d87a1cef6e8e0"
format_stamp: "Formatted at 2026-08-11 09:14:44 on dist-test-slave-ncp9"
server_key: "c4b56a2d27b48e60fe739bfd1d6e0ac4"
server_key_iv: "71c11f3670ac2133c590982e7aaf1b8c"
server_key_version: "encryptionkey@0"
I20260811 09:14:44.724251 24110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:14:44.727963 24131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:44.729277 24110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 09:14:44.729424 24110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "1b172a701cc64558bf5d87a1cef6e8e0"
format_stamp: "Formatted at 2026-08-11 09:14:44 on dist-test-slave-ncp9"
server_key: "c4b56a2d27b48e60fe739bfd1d6e0ac4"
server_key_iv: "71c11f3670ac2133c590982e7aaf1b8c"
server_key_version: "encryptionkey@0"
I20260811 09:14:44.729614 24110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:44.754554 24110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:44.758769 24110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:44.759055 24110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:44.759979 24110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:44.761466 24110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:44.761546 24110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:44.761667 24110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:44.761720 24110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:44.792305 24110 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:38841
I20260811 09:14:44.792361 24334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:38841 every 8 connection(s)
I20260811 09:14:44.793576 24110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:14:44.797475 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 24110
I20260811 09:14:44.797659 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:14:44.798209 17473 external_mini_cluster.cc:1428] Setting key ee9f40070d9ea44ad459b1d7374420ee
Aug 11 09:14:44 dist-test-slave-ncp9 krb5kdc[23669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439684, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:44.815224 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:44.795999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:38273 (local address 127.17.16.126:33477)
0811 09:14:44.796178 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:44.796182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:44.797399 (+ 1217us) server_negotiation.cc:407] Connection header received
0811 09:14:44.799716 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:44.799722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:44.799801 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:44.799907 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:44.801879 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:44.802602 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:44.804598 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:44.805074 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:44.805162 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:44.809128 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:44.809133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:44.809136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:44.809170 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:44.812195 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:44.812986 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:44.812991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:44.812994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:44.813152 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:44.813548 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:44.813553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:44.813557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:44.813892 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:44.814038 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:44.814681 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 09:14:44.814898 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:14:44.816044 24341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:44.796425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:33477 (local address 127.17.16.66:38273)
0811 09:14:44.797209 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:44.797264 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:44.799452 (+ 2188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:44.800156 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:44.800168 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:44.800923 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:44.801702 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:44.801729 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:44.802760 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:44.802765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:44.804466 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:44.804481 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:44.805287 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:44.805294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:44.805573 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:44.806310 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:44.806338 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:44.808862 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:44.812337 (+ 3475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:44.812345 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:44.812357 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:44.812825 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:44.813248 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:44.813251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:44.813254 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:44.813434 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:44.813998 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:44.814003 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:44.814313 (+ 310us) client_negotiation.cc:769] Sending connection context
0811 09:14:44.814532 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 09:14:44.814921 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":256,"threads_started":1}
I20260811 09:14:44.817422 24336 heartbeater.cc:347] Connected to a master server at 127.17.16.126:33477
I20260811 09:14:44.817719 24336 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:44.818467 24336 heartbeater.cc:511] Master 127.17.16.126:33477 requested a full tablet report, sending...
I20260811 09:14:44.819900 23750 ts_manager.cc:194] Registered new tserver with Master: 1b172a701cc64558bf5d87a1cef6e8e0 (127.17.16.66:38841)
I20260811 09:14:44.820693 23750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:38273
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:14:44.848855 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:33477
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:42375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:45.169962 24347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:45.170251 24347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:45.170378 24347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:45.177512 24347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:45.177626 24347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:45.177779 24347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:14:45.187348 24347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:45.188566 24347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:45.189821 24347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:45.202131 24360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:45.202185 24358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:45.202659 24367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:45.203102 24347 server_base.cc:1034] running on GCE node
I20260811 09:14:45.203648 24347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:45.204475 24347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:45.205678 24347 hybrid_clock.cc:648] HybridClock initialized: now 1786439685205623 us; error 47 us; skew 500 ppm
I20260811 09:14:45.205966 24347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:45 dist-test-slave-ncp9 krb5kdc[23669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439685, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:45.209028 24347 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:14:45.210388 24347 webserver.cc:460] Webserver started at http://127.17.16.67:44161/ using document root <none> and password file <none>
I20260811 09:14:45.211027 24347 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:45.211154 24347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:45.211438 24347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:45.214399 24347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "26f4273e84d04fb29751e33c8390c176"
format_stamp: "Formatted at 2026-08-11 09:14:45 on dist-test-slave-ncp9"
server_key: "298260121adc28ecebed6ff61cc42f80"
server_key_iv: "9fa2ef43521048a4c339355d0dcd3cfc"
server_key_version: "encryptionkey@0"
I20260811 09:14:45.215145 24347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "26f4273e84d04fb29751e33c8390c176"
format_stamp: "Formatted at 2026-08-11 09:14:45 on dist-test-slave-ncp9"
server_key: "298260121adc28ecebed6ff61cc42f80"
server_key_iv: "9fa2ef43521048a4c339355d0dcd3cfc"
server_key_version: "encryptionkey@0"
I20260811 09:14:45.220220 24347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:14:45.223665 24379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:45.224934 24347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:14:45.225088 24347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "26f4273e84d04fb29751e33c8390c176"
format_stamp: "Formatted at 2026-08-11 09:14:45 on dist-test-slave-ncp9"
server_key: "298260121adc28ecebed6ff61cc42f80"
server_key_iv: "9fa2ef43521048a4c339355d0dcd3cfc"
server_key_version: "encryptionkey@0"
I20260811 09:14:45.225270 24347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:45.244107 24347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:45.248086 24347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:45.248349 24347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:45.249286 24347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:45.250559 24347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:45.250635 24347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:45.250756 24347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:45.250816 24347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:45.280104 24347 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:37671
I20260811 09:14:45.280140 24571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:37671 every 8 connection(s)
I20260811 09:14:45.281419 24347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:14:45.287717 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 24347
I20260811 09:14:45.287879 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:14:45.288308 17473 external_mini_cluster.cc:1428] Setting key 03a84a3830f602c6c1c745dc36ee05aa
Aug 11 09:14:45 dist-test-slave-ncp9 krb5kdc[23669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439685, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:45.301190 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:45.283814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:34211 (local address 127.17.16.126:33477)
0811 09:14:45.283978 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:45.283982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:45.285130 (+ 1148us) server_negotiation.cc:407] Connection header received
0811 09:14:45.286931 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:45.286938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:45.287044 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:45.287133 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:45.289132 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.290176 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.292423 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.292828 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.292954 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:45.296236 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:45.296240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:45.296244 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:45.296274 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:45.298760 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.299385 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.299388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.299390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.299509 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.299838 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.299841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.299843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.300095 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:45.300207 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:45.300751 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 09:14:45.300941 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:14:45.302013 24577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:45.284237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:33477 (local address 127.17.16.67:34211)
0811 09:14:45.284955 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:45.284992 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:45.286638 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:45.287402 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:45.287411 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:45.288150 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:45.288937 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:45.288964 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.290353 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.290359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:45.292272 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:45.292287 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.293085 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.293094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:45.293356 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:45.293985 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:45.294010 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:45.296040 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:45.298880 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:45.298887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:45.298897 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:45.299294 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:45.299585 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:45.299588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:45.299590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:45.299753 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:45.300190 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:45.300195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:45.300459 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 09:14:45.300665 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 09:14:45.301057 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":264,"threads_started":1}
I20260811 09:14:45.303329 24574 heartbeater.cc:347] Connected to a master server at 127.17.16.126:33477
I20260811 09:14:45.303630 24574 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:45.304275 24574 heartbeater.cc:511] Master 127.17.16.126:33477 requested a full tablet report, sending...
I20260811 09:14:45.305545 23750 ts_manager.cc:194] Registered new tserver with Master: 26f4273e84d04fb29751e33c8390c176 (127.17.16.67:37671)
I20260811 09:14:45.306279 23750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:34211
I20260811 09:14:45.315505 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:14:45.321753 24095 heartbeater.cc:503] Master 127.17.16.126:33477 was elected leader, sending a full tablet report...
Aug 11 09:14:45 dist-test-slave-ncp9 krb5kdc[23669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:14:45 dist-test-slave-ncp9 krb5kdc[23669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:14:45.367412 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:45.352766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.17.16.65:32833)
0811 09:14:45.353410 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:45.353420 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:45.353453 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:14:45.353546 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:45.353553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:45.353780 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:45.353926 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:45.355379 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.356400 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.357681 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.358236 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.358356 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:45.362091 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:45.362102 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:45.362116 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:45.362154 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:45.364661 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.365330 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.365337 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.365339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.365468 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.365864 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.365868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.365870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.366207 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:45.366502 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:45.366817 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 09:14:45.367078 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":286,"threads_started":1}
Aug 11 09:14:45 dist-test-slave-ncp9 krb5kdc[23669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:45.383490 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:45.371542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50704 (local address 127.17.16.126:33477)
0811 09:14:45.371748 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:45.371754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:45.371882 (+ 128us) server_negotiation.cc:407] Connection header received
0811 09:14:45.372002 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:45.372006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:45.372080 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:45.372154 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:45.373375 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.374098 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.375199 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.375577 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.375659 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:45.379118 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:45.379123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:45.379126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:45.379155 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:45.381425 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.381958 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.381962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.381964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.382088 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.382366 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.382369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.382371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.382656 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:45.382841 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:45.383059 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 09:14:45.383211 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 09:14:45.384012 23750 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50704
I20260811 09:14:45.384958 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 23878
I20260811 09:14:45.397184 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:45.385751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37622 (local address 127.17.16.65:32833)
0811 09:14:45.385920 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:45.385925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:45.386185 (+ 260us) server_negotiation.cc:407] Connection header received
0811 09:14:45.386318 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:45.386323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:45.386418 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:45.386499 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:45.387627 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.388294 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.389508 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.389920 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.390024 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:45.391995 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:45.392000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:45.392004 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:45.392033 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:45.394504 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.395200 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.395207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.395210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.395345 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.395746 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.395750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.395753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.396026 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:45.396262 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:45.396603 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 09:14:45.396843 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:14:45.397753 24086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:45.481338 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 23878
I20260811 09:14:45.501734 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 24110
Aug 11 09:14:45 dist-test-slave-ncp9 krb5kdc[23669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:14:45.517185 24595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:45.502663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.17.16.66:38841)
0811 09:14:45.503200 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:45.503208 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:45.503233 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:14:45.503294 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:45.503300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:45.503507 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:45.503651 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:45.504877 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.505927 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.507175 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.507727 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.507851 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:45.511859 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:45.511871 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:45.511884 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:45.511916 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:45.514503 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.515118 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.515125 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.515127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.515247 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.515620 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.515624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.515626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.515956 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:45.516269 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:45.516565 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 09:14:45.516820 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":262,"threads_started":1}
I20260811 09:14:45.518450 24329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:45.598704 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 24110
I20260811 09:14:45.621109 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 24347
Aug 11 09:14:45 dist-test-slave-ncp9 krb5kdc[23669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:14:45.637036 24600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:45.622107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54442 (local address 127.17.16.67:37671)
0811 09:14:45.622719 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:45.622726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:45.622756 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:14:45.622816 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:45.622822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:45.623036 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:45.623159 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:45.624346 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.625520 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.626751 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:45.627503 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:45.627614 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:45.631754 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:45.631765 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:45.631780 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:45.631814 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:45.634555 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.635166 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.635172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.635175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.635294 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:45.635631 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:45.635635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:45.635638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:45.635970 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:45.636222 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:45.636448 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 09:14:45.636642 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":301,"threads_started":1}
I20260811 09:14:45.638273 24560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:45.725869 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 24347
I20260811 09:14:45.746189 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 23698
I20260811 09:14:45.747412 23820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:45.850231 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 23698
2026-08-11T09:14:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4023 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:14:45 dist-test-slave-ncp9 krb5kdc[24607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:14:45 dist-test-slave-ncp9 krb5kdc[24607](info): set up 2 sockets
Aug 11 09:14:45 dist-test-slave-ncp9 krb5kdc[24607](info): commencing operation
krb5kdc: starting...
W20260811 09:14:47.071835 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:14:47 dist-test-slave-ncp9 krb5kdc[24607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:14:47.282688 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41619
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:47.603948 24626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:47.604246 24626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:47.604389 24626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:47.611438 24626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:47.611546 24626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:47.611624 24626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:47.611708 24626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:47.611770 24626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:47.621346 24626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:47.622591 24626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:47.623852 24626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:47.635377 24634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:47.635339 24641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:47.635399 24636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:47.636085 24626 server_base.cc:1034] running on GCE node
I20260811 09:14:47.636812 24626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:47.638176 24626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:47.639395 24626 hybrid_clock.cc:648] HybridClock initialized: now 1786439687639347 us; error 39 us; skew 500 ppm
I20260811 09:14:47.639747 24626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:47 dist-test-slave-ncp9 krb5kdc[24607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439687, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:47.642786 24626 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:14:47.643935 24626 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:47.644037 24626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:47.644349 24626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:47.647287 24626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "c157c81d588c4a12b4ebb71c5e270368"
format_stamp: "Formatted at 2026-08-11 09:14:47 on dist-test-slave-ncp9"
server_key: "d6c369644ca13a75e05ef691b4d4fa66"
server_key_iv: "3cdfc8b8a6dd89aeb31c610637efe865"
server_key_version: "encryptionkey@0"
I20260811 09:14:47.648128 24626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "c157c81d588c4a12b4ebb71c5e270368"
format_stamp: "Formatted at 2026-08-11 09:14:47 on dist-test-slave-ncp9"
server_key: "d6c369644ca13a75e05ef691b4d4fa66"
server_key_iv: "3cdfc8b8a6dd89aeb31c610637efe865"
server_key_version: "encryptionkey@0"
I20260811 09:14:47.653275 24626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 09:14:47.656577 24648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:47.657840 24626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:14:47.658004 24626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "c157c81d588c4a12b4ebb71c5e270368"
format_stamp: "Formatted at 2026-08-11 09:14:47 on dist-test-slave-ncp9"
server_key: "d6c369644ca13a75e05ef691b4d4fa66"
server_key_iv: "3cdfc8b8a6dd89aeb31c610637efe865"
server_key_version: "encryptionkey@0"
I20260811 09:14:47.658206 24626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:47.676473 24626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:47.680546 24626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:47.680788 24626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:47.698040 24626 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:41619
I20260811 09:14:47.698060 24735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:41619 every 8 connection(s)
I20260811 09:14:47.698939 24626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:47.702665 24736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:47.702865 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 24626
I20260811 09:14:47.703087 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:47.703905 17473 external_mini_cluster.cc:1428] Setting key fce9434e668b105fca74dcbb9efed04c
I20260811 09:14:47.716506 24736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368: Bootstrap starting.
Aug 11 09:14:47 dist-test-slave-ncp9 krb5kdc[24607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:47.720422 24736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:47.721540 24736 log.cc:824] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:47.724665 24736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368: No bootstrap required, opened a new log
I20260811 09:14:47.727231 24743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:47.706139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60266 (local address 127.17.16.126:41619)
0811 09:14:47.706947 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:47.706959 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:47.707001 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:14:47.709016 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:47.709046 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:47.709484 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:47.709812 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:47.711567 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:47.712820 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:47.714575 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:47.715224 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:47.715355 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:47.720129 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:47.720137 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:47.720152 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:47.720196 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:47.723194 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:47.723969 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:47.723975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:47.723981 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:47.724140 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:47.724521 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:47.724524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:47.724526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:47.725153 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:47.725381 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:47.725832 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 09:14:47.726178 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":382,"threads_started":1}
I20260811 09:14:47.732380 24736 raft_consensus.cc:348] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c157c81d588c4a12b4ebb71c5e270368" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41619 } }
I20260811 09:14:47.732738 24736 raft_consensus.cc:374] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:47.732825 24736 raft_consensus.cc:729] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c157c81d588c4a12b4ebb71c5e270368, State: Initialized, Role: FOLLOWER
I20260811 09:14:47.733342 24736 consensus_queue.cc:260] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c157c81d588c4a12b4ebb71c5e270368" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41619 } }
I20260811 09:14:47.733561 24736 raft_consensus.cc:388] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:47.733683 24736 raft_consensus.cc:482] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:47.733824 24736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:47.735893 24736 raft_consensus.cc:504] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c157c81d588c4a12b4ebb71c5e270368" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41619 } }
I20260811 09:14:47.736348 24736 leader_election.cc:302] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c157c81d588c4a12b4ebb71c5e270368; no voters:
I20260811 09:14:47.736994 24736 leader_election.cc:288] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:47.737141 24749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:47.737753 24749 raft_consensus.cc:686] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [term 1 LEADER]: Becoming Leader. State: Replica: c157c81d588c4a12b4ebb71c5e270368, State: Running, Role: LEADER
I20260811 09:14:47.738268 24749 consensus_queue.cc:237] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c157c81d588c4a12b4ebb71c5e270368" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41619 } }
I20260811 09:14:47.739153 24736 sys_catalog.cc:564] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:47.741000 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c157c81d588c4a12b4ebb71c5e270368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c157c81d588c4a12b4ebb71c5e270368" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41619 } } }
I20260811 09:14:47.741333 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:47.742028 24752 sys_catalog.cc:455] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c157c81d588c4a12b4ebb71c5e270368. Latest consensus state: current_term: 1 leader_uuid: "c157c81d588c4a12b4ebb71c5e270368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c157c81d588c4a12b4ebb71c5e270368" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41619 } } }
I20260811 09:14:47.742394 24752 sys_catalog.cc:458] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:47.742857 24758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:47.748457 24758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:47.757057 24758 catalog_manager.cc:1347] Generated new cluster ID: 685476ebde1a4f8d8fb1913e98ce2845
I20260811 09:14:47.757170 24758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:47.793113 24758 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:47.794798 24758 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:47.803494 24758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368: Generated new TSK 0
I20260811 09:14:47.804466 24758 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:14:47.870059 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41619
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:48.197983 24787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:48.198266 24787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:48.198412 24787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:48.205567 24787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:48.205688 24787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:48.205845 24787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:14:48.215374 24787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:48.216657 24787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:48.218009 24787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:48.230297 24799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:48.230335 24803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:48.230360 24800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:48.231381 24787 server_base.cc:1034] running on GCE node
I20260811 09:14:48.231973 24787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:48.232909 24787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:48.234118 24787 hybrid_clock.cc:648] HybridClock initialized: now 1786439688234065 us; error 50 us; skew 500 ppm
I20260811 09:14:48.234403 24787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:48 dist-test-slave-ncp9 krb5kdc[24607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439688, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:48.237468 24787 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:14:48.238518 24787 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:48.238607 24787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:48.238881 24787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:48.241992 24787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "e125d513bf1148d484583a70e80413d0"
format_stamp: "Formatted at 2026-08-11 09:14:48 on dist-test-slave-ncp9"
server_key: "3d9f1f23c40e6335e2e2b90998a8b120"
server_key_iv: "9d82a2dc8c55d5ee7741a8168343b018"
server_key_version: "encryptionkey@0"
I20260811 09:14:48.242854 24787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "e125d513bf1148d484583a70e80413d0"
format_stamp: "Formatted at 2026-08-11 09:14:48 on dist-test-slave-ncp9"
server_key: "3d9f1f23c40e6335e2e2b90998a8b120"
server_key_iv: "9d82a2dc8c55d5ee7741a8168343b018"
server_key_version: "encryptionkey@0"
I20260811 09:14:48.247833 24787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:14:48.251232 24816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:48.252470 24787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:14:48.252662 24787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "e125d513bf1148d484583a70e80413d0"
format_stamp: "Formatted at 2026-08-11 09:14:48 on dist-test-slave-ncp9"
server_key: "3d9f1f23c40e6335e2e2b90998a8b120"
server_key_iv: "9d82a2dc8c55d5ee7741a8168343b018"
server_key_version: "encryptionkey@0"
I20260811 09:14:48.252851 24787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:48.271068 24787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:48.275099 24787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:48.275317 24787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:48.276196 24787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:48.277536 24787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:48.277637 24787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:48.277721 24787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:48.277776 24787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:48.308218 24787 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:34987
I20260811 09:14:48.308265 25032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:34987 every 8 connection(s)
I20260811 09:14:48.309100 24787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:14:48.313850 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 24787
I20260811 09:14:48.314030 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:14:48.314636 17473 external_mini_cluster.cc:1428] Setting key 17b53509ee24491fc8c89323b2829b0a
Aug 11 09:14:48 dist-test-slave-ncp9 krb5kdc[24607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439688, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:48.328943 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:48.311418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:36539 (local address 127.17.16.126:41619)
0811 09:14:48.311840 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:48.311845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:48.312733 (+ 888us) server_negotiation.cc:407] Connection header received
0811 09:14:48.314595 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:48.314601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:48.314687 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:48.314797 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:48.316561 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:48.317625 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:48.319195 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:48.319728 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:48.319846 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:48.323177 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:48.323184 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:48.323189 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:48.323226 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:48.325889 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:48.326779 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:48.326783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:48.326786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:48.326896 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:48.327327 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:48.327330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:48.327332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:48.327608 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:48.327715 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:48.328500 (+ 785us) server_negotiation.cc:299] Negotiation successful
0811 09:14:48.328663 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":229,"threads_started":1}
I20260811 09:14:48.330276 25042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:48.311888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41619 (local address 127.17.16.65:36539)
0811 09:14:48.312583 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:48.312622 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:48.314335 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:48.315057 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:48.315068 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:48.315642 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:48.316389 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:48.316410 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:48.317798 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:48.317802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:48.319027 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:48.319040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:48.319939 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:48.319944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:48.320149 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:48.320778 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:48.320800 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:48.322960 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:48.326039 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:48.326048 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:48.326065 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:48.326671 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:48.326990 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:48.326995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:48.326999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:48.327233 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:48.327732 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:48.327740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:48.328118 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 09:14:48.328428 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 09:14:48.329006 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":263,"threads_started":1}
I20260811 09:14:48.331982 25039 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41619
I20260811 09:14:48.332300 25039 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:48.333051 25039 heartbeater.cc:511] Master 127.17.16.126:41619 requested a full tablet report, sending...
I20260811 09:14:48.335269 24672 ts_manager.cc:194] Registered new tserver with Master: e125d513bf1148d484583a70e80413d0 (127.17.16.65:34987)
I20260811 09:14:48.336766 24672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:36539
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:14:48.365386 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41619
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:48.685627 25045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:48.685904 25045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:48.686054 25045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:48.693058 25045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:48.693164 25045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:48.693318 25045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:14:48.702893 25045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:48.704114 25045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:48.705476 25045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:48.717866 25066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:48.717892 25058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:48.717917 25053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:48.719061 25045 server_base.cc:1034] running on GCE node
I20260811 09:14:48.719679 25045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:48.720640 25045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:48.721856 25045 hybrid_clock.cc:648] HybridClock initialized: now 1786439688721793 us; error 51 us; skew 500 ppm
I20260811 09:14:48.722198 25045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:48 dist-test-slave-ncp9 krb5kdc[24607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439688, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:48.729094 25045 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:14:48.730206 25045 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:48.730319 25045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:48.730631 25045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:48.733558 25045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "68685fe275d74490953acf73bb3156af"
format_stamp: "Formatted at 2026-08-11 09:14:48 on dist-test-slave-ncp9"
server_key: "b7fef0d85bca1025c7d390033717b38b"
server_key_iv: "4763f3ec891d5b93e9ec972fa22aff7f"
server_key_version: "encryptionkey@0"
I20260811 09:14:48.734351 25045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "68685fe275d74490953acf73bb3156af"
format_stamp: "Formatted at 2026-08-11 09:14:48 on dist-test-slave-ncp9"
server_key: "b7fef0d85bca1025c7d390033717b38b"
server_key_iv: "4763f3ec891d5b93e9ec972fa22aff7f"
server_key_version: "encryptionkey@0"
I20260811 09:14:48.739409 25045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:14:48.742758 25078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:48.744077 25045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:14:48.744221 25045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "68685fe275d74490953acf73bb3156af"
format_stamp: "Formatted at 2026-08-11 09:14:48 on dist-test-slave-ncp9"
server_key: "b7fef0d85bca1025c7d390033717b38b"
server_key_iv: "4763f3ec891d5b93e9ec972fa22aff7f"
server_key_version: "encryptionkey@0"
I20260811 09:14:48.744448 25045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:48.758289 25045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:48.762046 25045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:48.762301 25045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:48.763253 25045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:48.764535 25045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:48.764623 25045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:48.764722 25045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:48.764787 25045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:48.794862 25045 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:35457
I20260811 09:14:48.794926 25274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:35457 every 8 connection(s)
I20260811 09:14:48.795701 25045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:14:48.796558 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 25045
I20260811 09:14:48.796756 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:14:48.797355 17473 external_mini_cluster.cc:1428] Setting key 9dd4daf271e03a0fedf9ba291d3d99a1
Aug 11 09:14:48 dist-test-slave-ncp9 krb5kdc[24607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439688, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:48.817749 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:48.798619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:33245 (local address 127.17.16.126:41619)
0811 09:14:48.798790 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:48.798794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:48.800284 (+ 1490us) server_negotiation.cc:407] Connection header received
0811 09:14:48.802769 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:48.802774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:48.802847 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:48.802931 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:48.804914 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:48.805646 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:48.807689 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:48.808116 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:48.808210 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:48.811739 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:48.811745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:48.811750 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:48.811789 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:48.814792 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:48.815447 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:48.815453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:48.815456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:48.815639 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:48.816012 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:48.816016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:48.816019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:48.816356 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:48.816502 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:48.817172 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 09:14:48.817404 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:14:48.818811 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:48.799180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41619 (local address 127.17.16.66:33245)
0811 09:14:48.800086 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:48.800151 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:48.802453 (+ 2302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:48.803185 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:48.803197 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:48.803913 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:48.804703 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:48.804730 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:48.805806 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:48.805811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:48.807541 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:48.807556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:48.808338 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:48.808345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:48.808589 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:48.809293 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:48.809321 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:48.811511 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:48.814914 (+ 3403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:48.814921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:48.814932 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:48.815334 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:48.815742 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:48.815745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:48.815747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:48.815912 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:48.816454 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:48.816459 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:48.816757 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 09:14:48.817040 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 09:14:48.817558 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":377,"threads_started":1}
I20260811 09:14:48.820309 25275 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41619
I20260811 09:14:48.820591 25275 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:48.821285 25275 heartbeater.cc:511] Master 127.17.16.126:41619 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
I20260811 09:14:48.822759 24672 ts_manager.cc:194] Registered new tserver with Master: 68685fe275d74490953acf73bb3156af (127.17.16.66:35457)
I20260811 09:14:48.823553 24672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:33245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:14:48.851881 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41619
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:49.176050 25285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:49.176337 25285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:49.176481 25285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:49.183740 25285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:49.183859 25285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:49.184036 25285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:14:49.193511 25285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:49.194766 25285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:49.196053 25285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:49.208247 25298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:49.208279 25301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:49.208302 25299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:49.208954 25285 server_base.cc:1034] running on GCE node
I20260811 09:14:49.209522 25285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:49.210392 25285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:49.211592 25285 hybrid_clock.cc:648] HybridClock initialized: now 1786439689211536 us; error 44 us; skew 500 ppm
I20260811 09:14:49.211879 25285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:49 dist-test-slave-ncp9 krb5kdc[24607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439689, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:49.214823 25285 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:14:49.215865 25285 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:49.215970 25285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:49.216269 25285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:49.219143 25285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "ddf28ae7f5c9498eab07d9a042d07aea"
format_stamp: "Formatted at 2026-08-11 09:14:49 on dist-test-slave-ncp9"
server_key: "bb8e2045d50d0ac4145c6fe40c85ca35"
server_key_iv: "9261029ecb8a8e36214a72135262c748"
server_key_version: "encryptionkey@0"
I20260811 09:14:49.219930 25285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "ddf28ae7f5c9498eab07d9a042d07aea"
format_stamp: "Formatted at 2026-08-11 09:14:49 on dist-test-slave-ncp9"
server_key: "bb8e2045d50d0ac4145c6fe40c85ca35"
server_key_iv: "9261029ecb8a8e36214a72135262c748"
server_key_version: "encryptionkey@0"
I20260811 09:14:49.224815 25285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 09:14:49.228068 25309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:49.229343 25285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 09:14:49.229512 25285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "ddf28ae7f5c9498eab07d9a042d07aea"
format_stamp: "Formatted at 2026-08-11 09:14:49 on dist-test-slave-ncp9"
server_key: "bb8e2045d50d0ac4145c6fe40c85ca35"
server_key_iv: "9261029ecb8a8e36214a72135262c748"
server_key_version: "encryptionkey@0"
I20260811 09:14:49.229712 25285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:49.262600 25285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:49.266546 25285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:49.266808 25285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:49.267791 25285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:49.269165 25285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:49.269264 25285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:49.269376 25285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:49.269446 25285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:49.299531 25285 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:46827
I20260811 09:14:49.299576 25505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:46827 every 8 connection(s)
I20260811 09:14:49.300371 25285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:14:49.301927 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 25285
I20260811 09:14:49.302083 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:14:49.302652 17473 external_mini_cluster.cc:1428] Setting key 91a40a6fff2720ee3e7645ce26afe01f
Aug 11 09:14:49 dist-test-slave-ncp9 krb5kdc[24607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439689, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:49.320247 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.302884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:44399 (local address 127.17.16.126:41619)
0811 09:14:49.303048 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.303053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.304485 (+ 1432us) server_negotiation.cc:407] Connection header received
0811 09:14:49.306481 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.306485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.306576 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.306657 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:49.308146 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.309307 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.310794 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.311432 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.311575 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.314780 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:49.314785 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:49.314789 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:49.314821 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:49.317472 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:49.318293 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:49.318297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:49.318299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:49.318432 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:49.318798 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:49.318801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:49.318803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:49.319044 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:49.319161 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.319841 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.320005 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:14:49.321166 25513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.303489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41619 (local address 127.17.16.67:44399)
0811 09:14:49.304298 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:49.304353 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:49.306242 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:49.306834 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:49.306841 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:49.307365 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:49.307970 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.307991 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.309443 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.309447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.310648 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.310661 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.311641 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.311646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.311824 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.312425 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:49.312448 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:49.314584 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:49.317608 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:49.317617 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:49.317630 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:49.318189 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:49.318540 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:49.318543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:49.318546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:49.318717 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:49.319172 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:49.319178 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:49.319483 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 09:14:49.319738 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 09:14:49.320108 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":313,"threads_started":1}
I20260811 09:14:49.322402 25507 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41619
I20260811 09:14:49.322693 25507 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:49.323290 25507 heartbeater.cc:511] Master 127.17.16.126:41619 requested a full tablet report, sending...
I20260811 09:14:49.324573 24672 ts_manager.cc:194] Registered new tserver with Master: ddf28ae7f5c9498eab07d9a042d07aea (127.17.16.67:46827)
I20260811 09:14:49.325381 24672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:44399
I20260811 09:14:49.330013 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:14:49.339862 25039 heartbeater.cc:503] Master 127.17.16.126:41619 was elected leader, sending a full tablet report...
I20260811 09:14:49.346208 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.334309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53892 (local address 127.17.16.126:41619)
0811 09:14:49.334456 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.334460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.334969 (+ 509us) server_negotiation.cc:407] Connection header received
0811 09:14:49.335124 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.335128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.335186 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.335256 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:49.336547 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.337281 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.338478 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.338916 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.339037 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.341638 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:49.341643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:49.341646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:49.341676 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:49.344087 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:49.344674 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:49.344678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:49.344680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:49.344803 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:49.345127 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:49.345130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:49.345132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:49.345410 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:49.345604 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.345818 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.345971 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 09:14:49.349836 24672 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:14:49.356554 24672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:49.390519 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.368483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:34987 (local address 127.0.0.1:46768)
0811 09:14:49.379061 (+ 10578us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:49.379119 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:49.379311 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:49.381907 (+ 2596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:49.381916 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:49.381950 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:49.382378 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.382395 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.385452 (+ 3057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.385460 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.387515 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.387531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.389518 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.389525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.389741 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.389794 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 09:14:49.389926 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 09:14:49.390112 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10270,"thread_start_us":16310,"threads_started":1}
I20260811 09:14:49.391566 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.368644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46768 (local address 127.17.16.65:34987)
0811 09:14:49.380917 (+ 12273us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.380928 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.380962 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:14:49.381089 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.381097 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.381362 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.381588 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:49.382836 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.384565 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.387682 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.389132 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.389284 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.389525 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.391046 (+ 1521us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.391223 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12058,"thread_start_us":13908,"threads_started":1}
I20260811 09:14:49.393883 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.368529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35457 (local address 127.0.0.1:50512)
0811 09:14:49.385135 (+ 16606us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:49.385161 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:49.385287 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:49.386877 (+ 1590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:49.386881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:49.386905 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:49.387314 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.387326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.388862 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.388868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.391768 (+ 2900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.391782 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.393342 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.393347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.393461 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.393484 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:14:49.393572 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 09:14:49.393656 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16469,"thread_start_us":16791,"threads_started":1}
I20260811 09:14:49.394119 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.369130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50512 (local address 127.17.16.66:35457)
0811 09:14:49.385792 (+ 16662us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.385800 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.385830 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:14:49.385924 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.385931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.386166 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.386356 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:49.387515 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.388711 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.391922 (+ 3211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.393157 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.393300 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.393474 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.393646 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.393809 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16490,"thread_start_us":17388,"threads_started":1}
I20260811 09:14:49.395581 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.371043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:46827 (local address 127.0.0.1:55586)
0811 09:14:49.385595 (+ 14552us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:49.385614 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:49.385743 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:49.386224 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:49.386228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:49.386251 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:49.386639 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.386649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.388503 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.388509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.390639 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.390655 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.392421 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.392428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.394172 (+ 1744us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.394980 (+ 808us) client_negotiation.cc:769] Sending connection context
0811 09:14:49.395091 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 09:14:49.395223 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14415,"mutex_wait_us":66,"thread_start_us":7562,"threads_started":1}
I20260811 09:14:49.395655 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.371381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55586 (local address 127.17.16.67:46827)
0811 09:14:49.382366 (+ 10985us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.382375 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.385723 (+ 3348us) server_negotiation.cc:407] Connection header received
0811 09:14:49.385876 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.385882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.386052 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.386209 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:49.386841 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.388346 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.390832 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.392224 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.392356 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.392584 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.395169 (+ 2585us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.395331 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10793,"thread_start_us":10641,"threads_started":1}
I20260811 09:14:49.395561 24912 tablet_service.cc:1467] Processing CreateTablet for tablet 5016e845c7994de5bed94ceff3edc161 (DEFAULT_TABLE table=test-table [id=e073da8ba2d0447ba75b387cf9e2d673]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:49.397316 24912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5016e845c7994de5bed94ceff3edc161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:49.397264 25160 tablet_service.cc:1467] Processing CreateTablet for tablet 5016e845c7994de5bed94ceff3edc161 (DEFAULT_TABLE table=test-table [id=e073da8ba2d0447ba75b387cf9e2d673]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:49.398373 25160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5016e845c7994de5bed94ceff3edc161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:49.399401 25395 tablet_service.cc:1467] Processing CreateTablet for tablet 5016e845c7994de5bed94ceff3edc161 (DEFAULT_TABLE table=test-table [id=e073da8ba2d0447ba75b387cf9e2d673]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:49.400695 25395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5016e845c7994de5bed94ceff3edc161. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:49.410555 25554 tablet_bootstrap.cc:492] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: Bootstrap starting.
I20260811 09:14:49.413627 25555 tablet_bootstrap.cc:492] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: Bootstrap starting.
I20260811 09:14:49.414253 25554 tablet_bootstrap.cc:654] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:49.415040 25556 tablet_bootstrap.cc:492] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: Bootstrap starting.
I20260811 09:14:49.415740 25554 log.cc:824] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:49.416970 25555 tablet_bootstrap.cc:654] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:49.417982 25556 tablet_bootstrap.cc:654] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:49.418208 25555 log.cc:824] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:49.419080 25554 tablet_bootstrap.cc:492] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: No bootstrap required, opened a new log
I20260811 09:14:49.419046 25556 log.cc:824] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:49.419343 25554 ts_tablet_manager.cc:1397] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260811 09:14:49.421504 25555 tablet_bootstrap.cc:492] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: No bootstrap required, opened a new log
I20260811 09:14:49.421785 25555 ts_tablet_manager.cc:1397] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260811 09:14:49.427546 25556 tablet_bootstrap.cc:492] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: No bootstrap required, opened a new log
I20260811 09:14:49.427490 25554 raft_consensus.cc:348] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 } } peers { permanent_uuid: "68685fe275d74490953acf73bb3156af" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35457 } } peers { permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 } }
I20260811 09:14:49.427906 25554 raft_consensus.cc:374] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:49.427907 25556 ts_tablet_manager.cc:1397] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260811 09:14:49.428007 25554 raft_consensus.cc:729] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68685fe275d74490953acf73bb3156af, State: Initialized, Role: FOLLOWER
I20260811 09:14:49.428582 25554 consensus_queue.cc:260] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 } } peers { permanent_uuid: "68685fe275d74490953acf73bb3156af" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35457 } } peers { permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 } }
I20260811 09:14:49.429697 25275 heartbeater.cc:503] Master 127.17.16.126:41619 was elected leader, sending a full tablet report...
I20260811 09:14:49.430346 25554 ts_tablet_manager.cc:1428] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260811 09:14:49.431879 25555 raft_consensus.cc:348] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 } } peers { permanent_uuid: "68685fe275d74490953acf73bb3156af" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35457 } } peers { permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 } }
I20260811 09:14:49.432320 25555 raft_consensus.cc:374] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:49.432422 25555 raft_consensus.cc:729] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e125d513bf1148d484583a70e80413d0, State: Initialized, Role: FOLLOWER
I20260811 09:14:49.433569 25555 consensus_queue.cc:260] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 } } peers { permanent_uuid: "68685fe275d74490953acf73bb3156af" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35457 } } peers { permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 } }
I20260811 09:14:49.435468 25555 ts_tablet_manager.cc:1428] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260811 09:14:49.438465 25556 raft_consensus.cc:348] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 } } peers { permanent_uuid: "68685fe275d74490953acf73bb3156af" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35457 } } peers { permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 } }
I20260811 09:14:49.438792 25556 raft_consensus.cc:374] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:49.438862 25556 raft_consensus.cc:729] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddf28ae7f5c9498eab07d9a042d07aea, State: Initialized, Role: FOLLOWER
I20260811 09:14:49.439366 25556 consensus_queue.cc:260] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 } } peers { permanent_uuid: "68685fe275d74490953acf73bb3156af" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35457 } } peers { permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 } }
I20260811 09:14:49.440294 25507 heartbeater.cc:503] Master 127.17.16.126:41619 was elected leader, sending a full tablet report...
I20260811 09:14:49.440687 25556 ts_tablet_manager.cc:1428] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260811 09:14:49.454948 25562 raft_consensus.cc:482] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:49.455221 25562 raft_consensus.cc:504] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 } } peers { permanent_uuid: "68685fe275d74490953acf73bb3156af" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35457 } } peers { permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 } }
I20260811 09:14:49.456828 25562 leader_election.cc:288] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e125d513bf1148d484583a70e80413d0 (127.17.16.65:34987), ddf28ae7f5c9498eab07d9a042d07aea (127.17.16.67:46827)
I20260811 09:14:49.463426 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.457414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:38237 (local address 127.17.16.65:34987)
0811 09:14:49.457627 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.457633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.458164 (+ 531us) server_negotiation.cc:407] Connection header received
0811 09:14:49.458355 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.458359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.458416 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.458486 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:49.459092 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.459841 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.461696 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.462521 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.462666 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.462751 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.463072 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.463197 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:14:49.463537 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.457247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:48527 (local address 127.17.16.67:46827)
0811 09:14:49.457440 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.457447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.458271 (+ 824us) server_negotiation.cc:407] Connection header received
0811 09:14:49.458446 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.458450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.458513 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.458582 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:14:49.459242 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.459941 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.461777 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.462655 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.462816 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.462888 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.463177 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.463298 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:14:49.463619 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.457298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:46827 (local address 127.17.16.66:48527)
0811 09:14:49.458175 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:49.458193 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:49.458317 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:49.458614 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:49.458618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:49.458655 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:49.459099 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.459110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.460074 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.460083 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.461594 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.461607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.462869 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.462876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.463033 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.463057 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:14:49.463137 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 09:14:49.463251 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":23,"thread_start_us":525,"threads_started":1}
I20260811 09:14:49.463891 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.457253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:34987 (local address 127.17.16.66:38237)
0811 09:14:49.458066 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:49.458083 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:49.458209 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:49.458516 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:49.458520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:49.458544 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:14:49.458954 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.458964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.459965 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.459969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.461538 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:49.461550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.462723 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.462730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:49.462920 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.462944 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:14:49.463028 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 09:14:49.463151 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":76,"thread_start_us":456,"threads_started":1}
I20260811 09:14:49.464612 25426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5016e845c7994de5bed94ceff3edc161" candidate_uuid: "68685fe275d74490953acf73bb3156af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" is_pre_election: true
I20260811 09:14:49.464803 24946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5016e845c7994de5bed94ceff3edc161" candidate_uuid: "68685fe275d74490953acf73bb3156af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e125d513bf1148d484583a70e80413d0" is_pre_election: true
I20260811 09:14:49.465101 25426 raft_consensus.cc:2454] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68685fe275d74490953acf73bb3156af in term 0.
I20260811 09:14:49.465313 24946 raft_consensus.cc:2454] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68685fe275d74490953acf73bb3156af in term 0.
I20260811 09:14:49.465749 25081 leader_election.cc:302] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68685fe275d74490953acf73bb3156af, ddf28ae7f5c9498eab07d9a042d07aea; no voters:
I20260811 09:14:49.466135 25562 raft_consensus.cc:2781] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:49.466286 25562 raft_consensus.cc:482] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:49.466387 25562 raft_consensus.cc:3037] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:49.468528 25562 raft_consensus.cc:504] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 } } peers { permanent_uuid: "68685fe275d74490953acf73bb3156af" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35457 } } peers { permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 } }
I20260811 09:14:49.469149 25562 leader_election.cc:288] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [CANDIDATE]: Term 1 election: Requested vote from peers e125d513bf1148d484583a70e80413d0 (127.17.16.65:34987), ddf28ae7f5c9498eab07d9a042d07aea (127.17.16.67:46827)
I20260811 09:14:49.469496 24946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5016e845c7994de5bed94ceff3edc161" candidate_uuid: "68685fe275d74490953acf73bb3156af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e125d513bf1148d484583a70e80413d0"
I20260811 09:14:49.469679 24946 raft_consensus.cc:3037] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:49.469635 25426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5016e845c7994de5bed94ceff3edc161" candidate_uuid: "68685fe275d74490953acf73bb3156af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf28ae7f5c9498eab07d9a042d07aea"
I20260811 09:14:49.469821 25426 raft_consensus.cc:3037] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:49.471925 25426 raft_consensus.cc:2454] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68685fe275d74490953acf73bb3156af in term 1.
I20260811 09:14:49.471987 24946 raft_consensus.cc:2454] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68685fe275d74490953acf73bb3156af in term 1.
I20260811 09:14:49.472285 25081 leader_election.cc:302] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68685fe275d74490953acf73bb3156af, ddf28ae7f5c9498eab07d9a042d07aea; no voters:
I20260811 09:14:49.472579 25562 raft_consensus.cc:2781] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:49.473019 25562 raft_consensus.cc:686] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 1 LEADER]: Becoming Leader. State: Replica: 68685fe275d74490953acf73bb3156af, State: Running, Role: LEADER
I20260811 09:14:49.473457 25562 consensus_queue.cc:237] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 } } peers { permanent_uuid: "68685fe275d74490953acf73bb3156af" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35457 } } peers { permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 } }
I20260811 09:14:49.477988 24670 catalog_manager.cc:5495] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af reported cstate change: term changed from 0 to 1, leader changed from <none> to 68685fe275d74490953acf73bb3156af (127.17.16.66). New cstate: current_term: 1 leader_uuid: "68685fe275d74490953acf73bb3156af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68685fe275d74490953acf73bb3156af" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 } health_report { overall_health: UNKNOWN } } }
I20260811 09:14:49.504503 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.499775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.17.16.66:35457)
0811 09:14:49.499950 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.499955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.500068 (+ 113us) server_negotiation.cc:407] Connection header received
0811 09:14:49.500259 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.500263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.500343 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.500459 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:14:49.500752 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.501508 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.502771 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.503149 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.503231 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.503611 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:14:49.504095 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:14:49.504198 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.504237 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.504300 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:14:49.512588 24946 raft_consensus.cc:1261] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0 [term 1 FOLLOWER]: Refusing update from remote peer 68685fe275d74490953acf73bb3156af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:49.512593 25426 raft_consensus.cc:1261] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea [term 1 FOLLOWER]: Refusing update from remote peer 68685fe275d74490953acf73bb3156af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:49.513759 25574 consensus_queue.cc:1035] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddf28ae7f5c9498eab07d9a042d07aea" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 46827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:49.514128 25562 consensus_queue.cc:1035] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [LEADER]: Connected to new peer: Peer: permanent_uuid: "e125d513bf1148d484583a70e80413d0" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 34987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:49.547006 24672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:49.547262 24672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:49.550849 24672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c157c81d588c4a12b4ebb71c5e270368: Sending DeleteTablet for 3 replicas of tablet 5016e845c7994de5bed94ceff3edc161
I20260811 09:14:49.551963 24912 tablet_service.cc:1514] Processing DeleteTablet for tablet 5016e845c7994de5bed94ceff3edc161 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:49 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:46768
I20260811 09:14:49.551999 25160 tablet_service.cc:1514] Processing DeleteTablet for tablet 5016e845c7994de5bed94ceff3edc161 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:49 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:50512
I20260811 09:14:49.553102 25395 tablet_service.cc:1514] Processing DeleteTablet for tablet 5016e845c7994de5bed94ceff3edc161 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:49 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:55586
I20260811 09:14:49.556492 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 24787
I20260811 09:14:49.564606 25612 tablet_replica.cc:330] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: stopping tablet replica
I20260811 09:14:49.565304 25612 raft_consensus.cc:2229] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:49.565963 25612 raft_consensus.cc:2258] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:49.568318 25612 ts_tablet_manager.cc:1905] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:49.568930 25608 tablet_replica.cc:330] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: stopping tablet replica
I20260811 09:14:49.569404 25608 raft_consensus.cc:2229] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:14:49.570152 25608 raft_consensus.cc:2258] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:49.572217 25608 ts_tablet_manager.cc:1905] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:49.575779 25612 ts_tablet_manager.cc:1918] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:14:49.575953 25612 log.cc:1196] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/5016e845c7994de5bed94ceff3edc161
I20260811 09:14:49.576475 25612 ts_tablet_manager.cc:1939] T 5016e845c7994de5bed94ceff3edc161 P e125d513bf1148d484583a70e80413d0: Deleting consensus metadata
I20260811 09:14:49.578352 24655 catalog_manager.cc:4841] TS e125d513bf1148d484583a70e80413d0 (127.17.16.65:34987): tablet 5016e845c7994de5bed94ceff3edc161 (table test-table [id=e073da8ba2d0447ba75b387cf9e2d673]) successfully deleted
I20260811 09:14:49.579694 25608 ts_tablet_manager.cc:1918] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:14:49.579838 25608 log.cc:1196] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/5016e845c7994de5bed94ceff3edc161
I20260811 09:14:49.580295 25608 ts_tablet_manager.cc:1939] T 5016e845c7994de5bed94ceff3edc161 P 68685fe275d74490953acf73bb3156af: Deleting consensus metadata
I20260811 09:14:49.581879 24652 catalog_manager.cc:4841] TS 68685fe275d74490953acf73bb3156af (127.17.16.66:35457): tablet 5016e845c7994de5bed94ceff3edc161 (table test-table [id=e073da8ba2d0447ba75b387cf9e2d673]) successfully deleted
I20260811 09:14:49.582342 25613 tablet_replica.cc:330] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: stopping tablet replica
I20260811 09:14:49.582705 25613 raft_consensus.cc:2229] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:49.583142 25613 raft_consensus.cc:2258] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:49.584789 25613 ts_tablet_manager.cc:1905] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:14:49 dist-test-slave-ncp9 krb5kdc[24607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:14:49.590116 25613 ts_tablet_manager.cc:1918] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:14:49.590281 25613 log.cc:1196] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/5016e845c7994de5bed94ceff3edc161
I20260811 09:14:49.590777 25613 ts_tablet_manager.cc:1939] T 5016e845c7994de5bed94ceff3edc161 P ddf28ae7f5c9498eab07d9a042d07aea: Deleting consensus metadata
I20260811 09:14:49.592304 24651 catalog_manager.cc:4841] TS ddf28ae7f5c9498eab07d9a042d07aea (127.17.16.67:46827): tablet 5016e845c7994de5bed94ceff3edc161 (table test-table [id=e073da8ba2d0447ba75b387cf9e2d673]) successfully deleted
I20260811 09:14:49.597330 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.558494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46784 (local address 127.17.16.65:34987)
0811 09:14:49.558678 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.558684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.579351 (+ 20667us) server_negotiation.cc:407] Connection header received
0811 09:14:49.579882 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.579887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.579987 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.580080 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:49.581890 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.583024 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.584864 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.585467 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.585596 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.590681 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:49.590700 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:49.590718 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:49.590758 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:49.594402 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:49.595025 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:49.595034 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:49.595037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:49.595207 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:49.595528 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:49.595534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:49.595537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:49.596009 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:49.596322 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.596488 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.596962 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:14:49.598006 25017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:49.704226 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 24787
I20260811 09:14:49.744413 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 25045
Aug 11 09:14:49 dist-test-slave-ncp9 krb5kdc[24607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:14:49.761615 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.745203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50522 (local address 127.17.16.66:35457)
0811 09:14:49.745396 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.745403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.745482 (+ 79us) server_negotiation.cc:407] Connection header received
0811 09:14:49.745687 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.745691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.745763 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.745836 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:49.747394 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.748250 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.750014 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.750451 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.750565 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.755662 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:49.755674 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:49.755689 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:49.755726 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:49.758754 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:49.759499 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:49.759506 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:49.759509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:49.759663 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:49.760033 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:49.760037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:49.760039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:49.760395 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:49.760670 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.761014 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.761256 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:14:49.762398 25267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:49.877295 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 25045
I20260811 09:14:49.902737 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 25285
Aug 11 09:14:49 dist-test-slave-ncp9 krb5kdc[24607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:14:49.917523 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:49.903469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55592 (local address 127.17.16.67:46827)
0811 09:14:49.903656 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:49.903663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:49.903725 (+ 62us) server_negotiation.cc:407] Connection header received
0811 09:14:49.903917 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:49.903921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:49.903998 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:49.904084 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:49.905413 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.906149 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.907392 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:49.907790 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:49.907901 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:49.912049 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:49.912066 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:49.912083 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:49.912119 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:49.914780 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:49.915421 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:49.915429 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:49.915432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:49.915580 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:49.915957 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:49.915962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:49.915964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:49.916331 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:49.916614 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:49.916979 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 09:14:49.917202 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:14:49.918181 25500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:50.004530 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 25285
I20260811 09:14:50.023916 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 24626
I20260811 09:14:50.025079 24729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:50.134759 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 24626
2026-08-11T09:14:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4286 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T09:14:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:50Z Disabled control of system clock
I20260811 09:14:50.214998 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:36271
--webserver_interface=127.17.16.126
--webserver_port=0
--builtin_ntp_servers=127.17.16.84:40167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:36271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 09:14:50.533978 25629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:50.534245 25629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:50.534368 25629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:50.541476 25629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:50.541579 25629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:50.541648 25629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:50.541735 25629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:50.541791 25629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:50.550678 25629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:40167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:36271
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:36271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:50.551898 25629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:50.553691 25629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:50.564924 25638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:50.564867 25644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:50.564828 25640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:50.565711 25629 server_base.cc:1034] running on GCE node
I20260811 09:14:50.566452 25629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:50.567821 25629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:50.568976 25629 hybrid_clock.cc:648] HybridClock initialized: now 1786439690568921 us; error 44 us; skew 500 ppm
I20260811 09:14:50.569310 25629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:14:50.571468 25629 webserver.cc:460] Webserver started at http://127.17.16.126:45089/ using document root <none> and password file <none>
I20260811 09:14:50.572156 25629 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:50.572254 25629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:50.572558 25629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:50.575366 25629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "054aa415b2874f0cb3ef180ca9ce8fd1"
format_stamp: "Formatted at 2026-08-11 09:14:50 on dist-test-slave-ncp9"
server_key: "cecfc17df9ae802e20b5966308805eb4"
server_key_iv: "6a9717ecb1675610326cd5b8ad4e2fa9"
server_key_version: "encryptionkey@0"
I20260811 09:14:50.576109 25629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "054aa415b2874f0cb3ef180ca9ce8fd1"
format_stamp: "Formatted at 2026-08-11 09:14:50 on dist-test-slave-ncp9"
server_key: "cecfc17df9ae802e20b5966308805eb4"
server_key_iv: "6a9717ecb1675610326cd5b8ad4e2fa9"
server_key_version: "encryptionkey@0"
I20260811 09:14:50.580986 25629 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260811 09:14:50.584318 25655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:50.585582 25629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:14:50.585747 25629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "054aa415b2874f0cb3ef180ca9ce8fd1"
format_stamp: "Formatted at 2026-08-11 09:14:50 on dist-test-slave-ncp9"
server_key: "cecfc17df9ae802e20b5966308805eb4"
server_key_iv: "6a9717ecb1675610326cd5b8ad4e2fa9"
server_key_version: "encryptionkey@0"
I20260811 09:14:50.585954 25629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:50.591264 25629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:50.591979 25629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:50.592243 25629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:50.608068 25629 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:36271
I20260811 09:14:50.608099 25743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:36271 every 8 connection(s)
I20260811 09:14:50.609308 25629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:50.612797 25745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:50.613072 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 25629
I20260811 09:14:50.613263 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:50.613900 17473 external_mini_cluster.cc:1428] Setting key e4e5eb57d384aa040a9fbc4922aa749e
I20260811 09:14:50.623245 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:50.615770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33418 (local address 127.17.16.126:36271)
0811 09:14:50.616831 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:50.616845 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:50.616923 (+ 78us) server_negotiation.cc:407] Connection header received
0811 09:14:50.619314 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:50.619343 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:50.619896 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:50.620332 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:50.621308 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:50.621316 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:50.621332 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:50.621349 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:50.621422 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:50.621444 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:50.621667 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:50.621798 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 09:14:50.622066 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":329,"threads_started":1}
I20260811 09:14:50.626619 25745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1: Bootstrap starting.
I20260811 09:14:50.629534 25745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:50.630482 25745 log.cc:824] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:50.633050 25745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1: No bootstrap required, opened a new log
I20260811 09:14:50.639892 25745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054aa415b2874f0cb3ef180ca9ce8fd1" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 36271 } }
I20260811 09:14:50.640245 25745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:50.640341 25745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 054aa415b2874f0cb3ef180ca9ce8fd1, State: Initialized, Role: FOLLOWER
I20260811 09:14:50.640977 25745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054aa415b2874f0cb3ef180ca9ce8fd1" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 36271 } }
I20260811 09:14:50.641212 25745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:50.641345 25745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:50.641523 25745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:50.643780 25745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054aa415b2874f0cb3ef180ca9ce8fd1" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 36271 } }
I20260811 09:14:50.644250 25745 leader_election.cc:302] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 054aa415b2874f0cb3ef180ca9ce8fd1; no voters:
I20260811 09:14:50.644806 25745 leader_election.cc:288] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:50.644955 25756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:50.645613 25756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [term 1 LEADER]: Becoming Leader. State: Replica: 054aa415b2874f0cb3ef180ca9ce8fd1, State: Running, Role: LEADER
I20260811 09:14:50.646135 25756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054aa415b2874f0cb3ef180ca9ce8fd1" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 36271 } }
I20260811 09:14:50.646799 25745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:50.648840 25757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "054aa415b2874f0cb3ef180ca9ce8fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054aa415b2874f0cb3ef180ca9ce8fd1" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 36271 } } }
I20260811 09:14:50.649236 25757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:50.648923 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 054aa415b2874f0cb3ef180ca9ce8fd1. Latest consensus state: current_term: 1 leader_uuid: "054aa415b2874f0cb3ef180ca9ce8fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "054aa415b2874f0cb3ef180ca9ce8fd1" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 36271 } } }
I20260811 09:14:50.650663 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:50.653548 25764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:50.658979 25764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:50.666549 25764 catalog_manager.cc:1347] Generated new cluster ID: 407b753ec2f048f89c26590a70f8e017
I20260811 09:14:50.666651 25764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:50.680811 25764 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:50.681295 25764 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:50.690068 25764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1: Generated new TSK 0
I20260811 09:14:50.690820 25764 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:14:50.697064 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:36271
--builtin_ntp_servers=127.17.16.84:40167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 09:14:51.021847 25785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:51.022122 25785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:51.022270 25785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:51.029405 25785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:51.029513 25785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:51.029676 25785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:14:51.039343 25785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:40167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:36271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:51.040650 25785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:51.042451 25785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:51.055039 25799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:51.055161 25794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:51.055153 25793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:51.055913 25785 server_base.cc:1034] running on GCE node
I20260811 09:14:51.056453 25785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:51.057395 25785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:51.058588 25785 hybrid_clock.cc:648] HybridClock initialized: now 1786439691058534 us; error 45 us; skew 500 ppm
I20260811 09:14:51.058876 25785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:14:51.061004 25785 webserver.cc:460] Webserver started at http://127.17.16.65:40271/ using document root <none> and password file <none>
I20260811 09:14:51.061643 25785 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:51.061744 25785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:51.062057 25785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:51.064792 25785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "7234d10aacd34d44806e9a298072cc0a"
format_stamp: "Formatted at 2026-08-11 09:14:51 on dist-test-slave-ncp9"
server_key: "be7c45bd5317a4bcac1e4e33515bd247"
server_key_iv: "151187a64e8c41c02de00b4d7f7b6bd7"
server_key_version: "encryptionkey@0"
I20260811 09:14:51.065647 25785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "7234d10aacd34d44806e9a298072cc0a"
format_stamp: "Formatted at 2026-08-11 09:14:51 on dist-test-slave-ncp9"
server_key: "be7c45bd5317a4bcac1e4e33515bd247"
server_key_iv: "151187a64e8c41c02de00b4d7f7b6bd7"
server_key_version: "encryptionkey@0"
I20260811 09:14:51.070567 25785 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 09:14:51.073737 25815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:51.074915 25785 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 09:14:51.075058 25785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "7234d10aacd34d44806e9a298072cc0a"
format_stamp: "Formatted at 2026-08-11 09:14:51 on dist-test-slave-ncp9"
server_key: "be7c45bd5317a4bcac1e4e33515bd247"
server_key_iv: "151187a64e8c41c02de00b4d7f7b6bd7"
server_key_version: "encryptionkey@0"
I20260811 09:14:51.075265 25785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:51.080837 25785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:51.081584 25785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:51.081804 25785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:51.082665 25785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:51.084005 25785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:51.084090 25785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:51.084174 25785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:51.084226 25785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:51.113229 25785 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:45759
I20260811 09:14:51.113283 26002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:45759 every 8 connection(s)
I20260811 09:14:51.114467 25785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:14:51.119397 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 25785
I20260811 09:14:51.119585 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:14:51.120199 17473 external_mini_cluster.cc:1428] Setting key 94566f97793d8e96863464197b71f86d
I20260811 09:14:51.122704 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:51.117010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:48803 (local address 127.17.16.126:36271)
0811 09:14:51.117178 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:51.117183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:51.118494 (+ 1311us) server_negotiation.cc:407] Connection header received
0811 09:14:51.120232 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:51.120238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:51.120306 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:51.120395 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:51.121747 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:51.121753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:51.121757 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:51.121777 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:51.121818 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:51.121834 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:51.121970 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:51.122355 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 09:14:51.122426 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 09:14:51.123343 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:36271
--builtin_ntp_servers=127.17.16.84:40167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 09:14:51.123667 26008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:51.117434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:36271 (local address 127.17.16.65:48803)
0811 09:14:51.118332 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:51.118367 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:51.119973 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:51.120635 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:51.120645 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:51.120745 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:51.121448 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:14:51.121450 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:51.121494 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:14:51.121578 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:51.122063 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:51.122075 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:51.122083 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:14:51.122422 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 09:14:51.122664 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":266,"threads_started":1}
I20260811 09:14:51.125159 26004 heartbeater.cc:347] Connected to a master server at 127.17.16.126:36271
I20260811 09:14:51.125469 26004 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:51.126196 26004 heartbeater.cc:511] Master 127.17.16.126:36271 requested a full tablet report, sending...
I20260811 09:14:51.128311 25683 ts_manager.cc:194] Registered new tserver with Master: 7234d10aacd34d44806e9a298072cc0a (127.17.16.65:45759)
W20260811 09:14:51.451020 26009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:51.451283 26009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:51.451412 26009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:51.458964 26009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:51.459064 26009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:51.459205 26009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:14:51.468396 26009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:40167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:36271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:51.469668 26009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:51.471401 26009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:51.483930 26022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:51.483881 26027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:51.483955 26024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:51.485123 26009 server_base.cc:1034] running on GCE node
I20260811 09:14:51.485764 26009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:51.486677 26009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:51.487875 26009 hybrid_clock.cc:648] HybridClock initialized: now 1786439691487818 us; error 46 us; skew 500 ppm
I20260811 09:14:51.488173 26009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:14:51.490399 26009 webserver.cc:460] Webserver started at http://127.17.16.66:45903/ using document root <none> and password file <none>
I20260811 09:14:51.491091 26009 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:51.491179 26009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:51.491468 26009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:51.494395 26009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "1e482c3dc9654ae6bb0305b47a4ffed1"
format_stamp: "Formatted at 2026-08-11 09:14:51 on dist-test-slave-ncp9"
server_key: "1e959d6c88a42be461543b2a4e988ede"
server_key_iv: "25b7874b6fb28df76dcd8f8d68f41915"
server_key_version: "encryptionkey@0"
I20260811 09:14:51.495203 26009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "1e482c3dc9654ae6bb0305b47a4ffed1"
format_stamp: "Formatted at 2026-08-11 09:14:51 on dist-test-slave-ncp9"
server_key: "1e959d6c88a42be461543b2a4e988ede"
server_key_iv: "25b7874b6fb28df76dcd8f8d68f41915"
server_key_version: "encryptionkey@0"
I20260811 09:14:51.500384 26009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:14:51.503906 26036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:51.505260 26009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:14:51.505416 26009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "1e482c3dc9654ae6bb0305b47a4ffed1"
format_stamp: "Formatted at 2026-08-11 09:14:51 on dist-test-slave-ncp9"
server_key: "1e959d6c88a42be461543b2a4e988ede"
server_key_iv: "25b7874b6fb28df76dcd8f8d68f41915"
server_key_version: "encryptionkey@0"
I20260811 09:14:51.505627 26009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:51.511021 26009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:51.511765 26009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:51.512003 26009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:51.512853 26009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:51.514230 26009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:51.514325 26009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:51.514432 26009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:51.514501 26009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:51.544183 26009 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:46647
I20260811 09:14:51.544234 26233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:46647 every 8 connection(s)
I20260811 09:14:51.545497 26009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:14:51.553433 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:51.547946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:58997 (local address 127.17.16.126:36271)
0811 09:14:51.548189 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:51.548196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:51.549531 (+ 1335us) server_negotiation.cc:407] Connection header received
0811 09:14:51.551482 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:51.551486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:51.551543 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:51.551633 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:51.552817 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:51.552823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:51.552827 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:51.552844 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:51.552875 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:51.552887 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:51.553026 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:51.553157 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 09:14:51.553207 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 09:14:51.554307 26239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:51.548468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:36271 (local address 127.17.16.66:58997)
0811 09:14:51.549381 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:51.549426 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:51.551193 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:51.551820 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:51.551827 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:51.551910 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:51.552534 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:14:51.552537 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:51.552574 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:14:51.552658 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:51.553003 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:51.553009 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:51.553014 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 09:14:51.553228 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 09:14:51.553409 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":257,"threads_started":1}
I20260811 09:14:51.554832 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 26009
I20260811 09:14:51.555015 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:14:51.555573 17473 external_mini_cluster.cc:1428] Setting key 34bfb746a28e01ce4b7e110064b2a4f4
I20260811 09:14:51.555730 26234 heartbeater.cc:347] Connected to a master server at 127.17.16.126:36271
I20260811 09:14:51.556034 26234 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:51.556726 26234 heartbeater.cc:511] Master 127.17.16.126:36271 requested a full tablet report, sending...
I20260811 09:14:51.558223 25683 ts_manager.cc:194] Registered new tserver with Master: 1e482c3dc9654ae6bb0305b47a4ffed1 (127.17.16.66:46647)
I20260811 09:14:51.558637 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:36271
--builtin_ntp_servers=127.17.16.84:40167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 09:14:51.890133 26240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:51.890404 26240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:51.890509 26240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:51.897586 26240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:51.897678 26240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:51.897791 26240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:14:51.906497 26240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:40167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:36271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:51.907728 26240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:51.909467 26240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:51.922010 26249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:51.922073 26253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:51.922094 26250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:51.923117 26240 server_base.cc:1034] running on GCE node
I20260811 09:14:51.923703 26240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:51.924621 26240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:51.925844 26240 hybrid_clock.cc:648] HybridClock initialized: now 1786439691925798 us; error 31 us; skew 500 ppm
I20260811 09:14:51.926185 26240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:14:51.928364 26240 webserver.cc:460] Webserver started at http://127.17.16.67:37757/ using document root <none> and password file <none>
I20260811 09:14:51.929095 26240 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:51.929203 26240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:51.929546 26240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:51.932523 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "538abc6dc8fa47658ccd54168f001195"
format_stamp: "Formatted at 2026-08-11 09:14:51 on dist-test-slave-ncp9"
server_key: "c0aa0a11075c7486dbed1419144883f8"
server_key_iv: "3ed4a0a2e2308cd725363d2cc52bc37d"
server_key_version: "encryptionkey@0"
I20260811 09:14:51.933352 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "538abc6dc8fa47658ccd54168f001195"
format_stamp: "Formatted at 2026-08-11 09:14:51 on dist-test-slave-ncp9"
server_key: "c0aa0a11075c7486dbed1419144883f8"
server_key_iv: "3ed4a0a2e2308cd725363d2cc52bc37d"
server_key_version: "encryptionkey@0"
I20260811 09:14:51.938695 26240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:14:51.942163 26266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:51.943560 26240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:14:51.943744 26240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "538abc6dc8fa47658ccd54168f001195"
format_stamp: "Formatted at 2026-08-11 09:14:51 on dist-test-slave-ncp9"
server_key: "c0aa0a11075c7486dbed1419144883f8"
server_key_iv: "3ed4a0a2e2308cd725363d2cc52bc37d"
server_key_version: "encryptionkey@0"
I20260811 09:14:51.943981 26240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:51.949824 26240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:51.950739 26240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:51.951009 26240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:51.951924 26240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:51.953385 26240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:51.953480 26240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:51.953583 26240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:51.953647 26240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:51.983342 26240 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:41395
I20260811 09:14:51.983397 26459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:41395 every 8 connection(s)
I20260811 09:14:51.984629 26240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:14:51.990545 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 26240
I20260811 09:14:51.990720 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:14:51.991134 17473 external_mini_cluster.cc:1428] Setting key ea80203b2d765eacf1c73e333e62a9d2
I20260811 09:14:51.992834 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:51.987139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:45353 (local address 127.17.16.126:36271)
0811 09:14:51.987338 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:51.987342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:51.988720 (+ 1378us) server_negotiation.cc:407] Connection header received
0811 09:14:51.990427 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:51.990433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:51.990497 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:51.990584 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:51.992123 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:51.992127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:51.992131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:51.992154 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:51.992187 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:51.992199 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:51.992309 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:51.992509 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 09:14:51.992576 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:14:51.993798 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:51.987592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:36271 (local address 127.17.16.67:45353)
0811 09:14:51.988562 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:51.988601 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:51.990164 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:51.990820 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:51.990830 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:51.990923 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:51.991598 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:14:51.991601 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:51.991648 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:14:51.991758 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:51.992304 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:51.992316 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:51.992324 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:14:51.992574 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 09:14:51.992817 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":271,"threads_started":1}
I20260811 09:14:51.995013 26460 heartbeater.cc:347] Connected to a master server at 127.17.16.126:36271
I20260811 09:14:51.995268 26460 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:51.995813 26460 heartbeater.cc:511] Master 127.17.16.126:36271 requested a full tablet report, sending...
I20260811 09:14:51.997100 25683 ts_manager.cc:194] Registered new tserver with Master: 538abc6dc8fa47658ccd54168f001195 (127.17.16.67:41395)
I20260811 09:14:52.006699 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:14:52.013566 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.011227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33428 (local address 127.17.16.126:36271)
0811 09:14:52.011406 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:52.011412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:52.011920 (+ 508us) server_negotiation.cc:407] Connection header received
0811 09:14:52.012060 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:52.012063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:52.012112 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:52.012186 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:52.013069 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:52.013073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:52.013076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:52.013089 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:52.013117 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:52.013128 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:52.013216 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:52.013268 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 09:14:52.013309 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:14:52.017222 25683 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:14:52.023787 25683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:14:52.046051 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.036169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:46647 (local address 127.0.0.1:34618)
0811 09:14:52.037575 (+ 1406us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.037635 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:52.037774 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:52.042784 (+ 5010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:52.042794 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:52.042887 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:52.043765 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:14:52.043769 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:52.043801 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:14:52.043892 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:52.045312 (+ 1420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:52.045325 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:52.045334 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 09:14:52.045498 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 09:14:52.045589 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1137,"thread_start_us":965,"threads_started":1}
I20260811 09:14:52.046944 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.036768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.17.16.66:46647)
0811 09:14:52.042172 (+ 5404us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:52.042181 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:52.042227 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:14:52.042303 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:52.042312 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:52.042556 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:52.043414 (+ 858us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:52.045036 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:52.045046 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:52.045061 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:52.045086 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:52.045143 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:52.045170 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:52.046439 (+ 1269us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:52.046517 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 09:14:52.046605 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5201,"thread_start_us":4883,"threads_started":1}
I20260811 09:14:52.048339 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.038991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:41395 (local address 127.0.0.1:54686)
0811 09:14:52.045326 (+ 6335us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.045343 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:52.045444 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:52.045977 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:52.045983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:52.046009 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:52.046854 (+ 845us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:14:52.046857 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:52.046873 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:14:52.046917 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:52.047714 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:52.047719 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:52.047723 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 09:14:52.047868 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 09:14:52.047927 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6217,"thread_start_us":6078,"threads_started":1}
I20260811 09:14:52.049078 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.039573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.17.16.67:41395)
0811 09:14:52.040415 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:52.040425 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:52.045454 (+ 5029us) server_negotiation.cc:407] Connection header received
0811 09:14:52.045572 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:52.045581 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:52.045816 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:52.045970 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:52.047055 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:52.047064 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:52.047076 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:52.047093 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:52.047145 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:52.047165 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:52.047352 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:52.048583 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0811 09:14:52.048687 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":465,"threads_started":1}
I20260811 09:14:52.049683 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.036169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45759 (local address 127.0.0.1:41870)
0811 09:14:52.041344 (+ 5175us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.041365 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:52.041467 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:52.047771 (+ 6304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:52.047776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:52.047797 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:52.048478 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:14:52.048481 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:52.048497 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:14:52.048549 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:52.049106 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:52.049111 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:52.049117 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 09:14:52.049242 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 09:14:52.049292 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5026,"mutex_wait_us":372,"thread_start_us":4501,"threads_started":1}
I20260811 09:14:52.050302 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.036621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.17.16.65:45759)
0811 09:14:52.045942 (+ 9321us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:52.045950 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:52.045976 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:14:52.047351 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:52.047359 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:52.047569 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:52.047714 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:52.048833 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:52.048841 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:52.048853 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:52.048877 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:52.048953 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:52.048974 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:52.049908 (+ 934us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:52.049983 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 09:14:52.050064 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9159,"thread_start_us":9023,"threads_started":1}
I20260811 09:14:52.051548 26124 tablet_service.cc:1467] Processing CreateTablet for tablet a28963da186f4381803ff713225acd93 (DEFAULT_TABLE table=test-table [id=c7cab00ce6bd4f4f8996d2b32750180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:52.053097 26124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a28963da186f4381803ff713225acd93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:52.052675 26347 tablet_service.cc:1467] Processing CreateTablet for tablet a28963da186f4381803ff713225acd93 (DEFAULT_TABLE table=test-table [id=c7cab00ce6bd4f4f8996d2b32750180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:52.054057 26347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a28963da186f4381803ff713225acd93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:52.055599 25901 tablet_service.cc:1467] Processing CreateTablet for tablet a28963da186f4381803ff713225acd93 (DEFAULT_TABLE table=test-table [id=c7cab00ce6bd4f4f8996d2b32750180c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:14:52.056597 25901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a28963da186f4381803ff713225acd93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:52.066896 26493 tablet_bootstrap.cc:492] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a: Bootstrap starting.
I20260811 09:14:52.069919 26493 tablet_bootstrap.cc:654] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:52.070844 26493 log.cc:824] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:52.073015 26496 tablet_bootstrap.cc:492] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: Bootstrap starting.
I20260811 09:14:52.073190 26495 tablet_bootstrap.cc:492] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: Bootstrap starting.
I20260811 09:14:52.073936 26493 tablet_bootstrap.cc:492] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a: No bootstrap required, opened a new log
I20260811 09:14:52.074182 26493 ts_tablet_manager.cc:1397] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 09:14:52.076392 26495 tablet_bootstrap.cc:654] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:52.076450 26496 tablet_bootstrap.cc:654] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:52.077637 26496 log.cc:824] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:52.077622 26495 log.cc:824] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:52.080431 26496 tablet_bootstrap.cc:492] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: No bootstrap required, opened a new log
I20260811 09:14:52.080663 26496 ts_tablet_manager.cc:1397] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 09:14:52.081673 26495 tablet_bootstrap.cc:492] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: No bootstrap required, opened a new log
I20260811 09:14:52.081915 26495 ts_tablet_manager.cc:1397] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260811 09:14:52.082664 26493 raft_consensus.cc:348] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 } } peers { permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 } } peers { permanent_uuid: "538abc6dc8fa47658ccd54168f001195" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 41395 } }
I20260811 09:14:52.083081 26493 raft_consensus.cc:374] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:52.083189 26493 raft_consensus.cc:729] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7234d10aacd34d44806e9a298072cc0a, State: Initialized, Role: FOLLOWER
I20260811 09:14:52.083877 26493 consensus_queue.cc:260] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 } } peers { permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 } } peers { permanent_uuid: "538abc6dc8fa47658ccd54168f001195" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 41395 } }
I20260811 09:14:52.085049 26004 heartbeater.cc:503] Master 127.17.16.126:36271 was elected leader, sending a full tablet report...
I20260811 09:14:52.085615 26493 ts_tablet_manager.cc:1428] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 09:14:52.088636 26496 raft_consensus.cc:348] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 } } peers { permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 } } peers { permanent_uuid: "538abc6dc8fa47658ccd54168f001195" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 41395 } }
I20260811 09:14:52.089036 26496 raft_consensus.cc:374] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:52.089140 26496 raft_consensus.cc:729] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 538abc6dc8fa47658ccd54168f001195, State: Initialized, Role: FOLLOWER
I20260811 09:14:52.089841 26496 consensus_queue.cc:260] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 } } peers { permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 } } peers { permanent_uuid: "538abc6dc8fa47658ccd54168f001195" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 41395 } }
I20260811 09:14:52.091046 26460 heartbeater.cc:503] Master 127.17.16.126:36271 was elected leader, sending a full tablet report...
I20260811 09:14:52.091200 26495 raft_consensus.cc:348] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 } } peers { permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 } } peers { permanent_uuid: "538abc6dc8fa47658ccd54168f001195" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 41395 } }
I20260811 09:14:52.091598 26495 raft_consensus.cc:374] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:52.091698 26495 raft_consensus.cc:729] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e482c3dc9654ae6bb0305b47a4ffed1, State: Initialized, Role: FOLLOWER
I20260811 09:14:52.091727 26496 ts_tablet_manager.cc:1428] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260811 09:14:52.092263 26495 consensus_queue.cc:260] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 } } peers { permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 } } peers { permanent_uuid: "538abc6dc8fa47658ccd54168f001195" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 41395 } }
I20260811 09:14:52.093246 26234 heartbeater.cc:503] Master 127.17.16.126:36271 was elected leader, sending a full tablet report...
I20260811 09:14:52.093838 26495 ts_tablet_manager.cc:1428] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
W20260811 09:14:52.116456 26007 tablet.cc:2369] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:14:52.199193 26504 raft_consensus.cc:482] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:14:52.199481 26504 raft_consensus.cc:504] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 } } peers { permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 } } peers { permanent_uuid: "538abc6dc8fa47658ccd54168f001195" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 41395 } }
I20260811 09:14:52.201118 26504 leader_election.cc:288] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7234d10aacd34d44806e9a298072cc0a (127.17.16.65:45759), 1e482c3dc9654ae6bb0305b47a4ffed1 (127.17.16.66:46647)
I20260811 09:14:52.203842 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.201444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45759 (local address 127.17.16.67:52303)
0811 09:14:52.201768 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.201791 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:52.201931 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:52.202320 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:52.202323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:52.202343 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:52.203000 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:14:52.203002 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:52.203017 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:14:52.203060 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:52.203422 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:52.203427 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:52.203432 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 09:14:52.203535 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 09:14:52.203591 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260811 09:14:52.203855 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.201610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:52303 (local address 127.17.16.65:45759)
0811 09:14:52.201802 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:52.201809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:52.201890 (+ 81us) server_negotiation.cc:407] Connection header received
0811 09:14:52.202063 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:52.202068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:52.202138 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:52.202269 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:52.203213 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:52.203220 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:52.203225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:52.203248 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:52.203291 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:52.203308 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:52.203428 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:52.203540 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 09:14:52.203598 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:14:52.204105 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.201610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:51649 (local address 127.17.16.66:46647)
0811 09:14:52.201807 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:52.201813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:52.202444 (+ 631us) server_negotiation.cc:407] Connection header received
0811 09:14:52.202546 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:52.202551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:52.202613 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:52.202694 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:52.203512 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:52.203518 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:52.203522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:52.203541 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:52.203573 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:52.203586 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:52.203694 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:52.203786 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 09:14:52.203846 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 09:14:52.204289 26510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.201492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:46647 (local address 127.17.16.67:51649)
0811 09:14:52.202328 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.202346 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:52.202454 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:52.202748 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:52.202753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:52.202775 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:52.203340 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:14:52.203342 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:52.203353 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:14:52.203376 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:52.203673 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:52.203678 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:52.203682 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 09:14:52.203798 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 09:14:52.203848 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":459,"threads_started":1}
I20260811 09:14:52.205042 25935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a28963da186f4381803ff713225acd93" candidate_uuid: "538abc6dc8fa47658ccd54168f001195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7234d10aacd34d44806e9a298072cc0a" is_pre_election: true
I20260811 09:14:52.205295 26164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a28963da186f4381803ff713225acd93" candidate_uuid: "538abc6dc8fa47658ccd54168f001195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" is_pre_election: true
I20260811 09:14:52.205693 25935 raft_consensus.cc:2454] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 538abc6dc8fa47658ccd54168f001195 in term 0.
I20260811 09:14:52.205821 26164 raft_consensus.cc:2454] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 538abc6dc8fa47658ccd54168f001195 in term 0.
I20260811 09:14:52.206367 26271 leader_election.cc:302] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 538abc6dc8fa47658ccd54168f001195, 7234d10aacd34d44806e9a298072cc0a; no voters:
I20260811 09:14:52.206725 26504 raft_consensus.cc:2781] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:14:52.206853 26504 raft_consensus.cc:482] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:14:52.206954 26504 raft_consensus.cc:3037] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:52.209268 26504 raft_consensus.cc:504] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 } } peers { permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 } } peers { permanent_uuid: "538abc6dc8fa47658ccd54168f001195" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 41395 } }
I20260811 09:14:52.209852 26504 leader_election.cc:288] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [CANDIDATE]: Term 1 election: Requested vote from peers 7234d10aacd34d44806e9a298072cc0a (127.17.16.65:45759), 1e482c3dc9654ae6bb0305b47a4ffed1 (127.17.16.66:46647)
I20260811 09:14:52.210300 25935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a28963da186f4381803ff713225acd93" candidate_uuid: "538abc6dc8fa47658ccd54168f001195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7234d10aacd34d44806e9a298072cc0a"
I20260811 09:14:52.210330 26164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a28963da186f4381803ff713225acd93" candidate_uuid: "538abc6dc8fa47658ccd54168f001195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1"
I20260811 09:14:52.210481 25935 raft_consensus.cc:3037] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:52.210512 26164 raft_consensus.cc:3037] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:52.212628 25935 raft_consensus.cc:2454] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 538abc6dc8fa47658ccd54168f001195 in term 1.
I20260811 09:14:52.212631 26164 raft_consensus.cc:2454] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 538abc6dc8fa47658ccd54168f001195 in term 1.
I20260811 09:14:52.213002 26271 leader_election.cc:302] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 538abc6dc8fa47658ccd54168f001195, 7234d10aacd34d44806e9a298072cc0a; no voters:
I20260811 09:14:52.213256 26504 raft_consensus.cc:2781] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:52.213661 26504 raft_consensus.cc:686] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 1 LEADER]: Becoming Leader. State: Replica: 538abc6dc8fa47658ccd54168f001195, State: Running, Role: LEADER
I20260811 09:14:52.214092 26504 consensus_queue.cc:237] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 } } peers { permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 } } peers { permanent_uuid: "538abc6dc8fa47658ccd54168f001195" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 41395 } }
I20260811 09:14:52.218767 25683 catalog_manager.cc:5495] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 reported cstate change: term changed from 0 to 1, leader changed from <none> to 538abc6dc8fa47658ccd54168f001195 (127.17.16.67). New cstate: current_term: 1 leader_uuid: "538abc6dc8fa47658ccd54168f001195" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "538abc6dc8fa47658ccd54168f001195" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 41395 } health_report { overall_health: HEALTHY } } }
I20260811 09:14:52.230718 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.229114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54700 (local address 127.17.16.67:41395)
0811 09:14:52.229281 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:52.229285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:52.229302 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:14:52.229377 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:52.229380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:52.229453 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:52.229532 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:52.230244 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:52.230248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:52.230251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:52.230261 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:52.230284 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:52.230294 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:52.230382 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:52.230442 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 09:14:52.230484 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 09:14:52.236182 26464 tablet.cc:2369] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:14:52.238416 26164 raft_consensus.cc:1261] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1 [term 1 FOLLOWER]: Refusing update from remote peer 538abc6dc8fa47658ccd54168f001195: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:52.238420 25935 raft_consensus.cc:1261] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a [term 1 FOLLOWER]: Refusing update from remote peer 538abc6dc8fa47658ccd54168f001195: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:14:52.239291 26512 consensus_queue.cc:1035] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e482c3dc9654ae6bb0305b47a4ffed1" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 46647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:52.239610 26504 consensus_queue.cc:1035] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7234d10aacd34d44806e9a298072cc0a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:14:52.250195 26519 mvcc.cc:205] Tried to move back new op lower bound from 7317256979398782976 to 7317256979312619520. Current Snapshot: MvccSnapshot[applied={T|T < 7317256979398782976}]
I20260811 09:14:52.251541 26520 mvcc.cc:205] Tried to move back new op lower bound from 7317256979398782976 to 7317256979312619520. Current Snapshot: MvccSnapshot[applied={T|T < 7317256979398782976}]
I20260811 09:14:52.262626 25679 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:52.262920 25679 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:14:52.266692 25679 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 054aa415b2874f0cb3ef180ca9ce8fd1: Sending DeleteTablet for 3 replicas of tablet a28963da186f4381803ff713225acd93
I20260811 09:14:52.267763 25901 tablet_service.cc:1514] Processing DeleteTablet for tablet a28963da186f4381803ff713225acd93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:52 UTC) from {username='slave'} at 127.0.0.1:41870
I20260811 09:14:52.267769 26124 tablet_service.cc:1514] Processing DeleteTablet for tablet a28963da186f4381803ff713225acd93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:52 UTC) from {username='slave'} at 127.0.0.1:34618
I20260811 09:14:52.267853 26347 tablet_service.cc:1514] Processing DeleteTablet for tablet a28963da186f4381803ff713225acd93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:14:52 UTC) from {username='slave'} at 127.0.0.1:54686
I20260811 09:14:52.272723 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 25785
I20260811 09:14:52.280573 26535 tablet_replica.cc:330] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: stopping tablet replica
I20260811 09:14:52.280795 26536 tablet_replica.cc:330] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a: stopping tablet replica
I20260811 09:14:52.281206 26536 raft_consensus.cc:2229] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:52.281209 26535 raft_consensus.cc:2229] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:14:52.281764 26536 raft_consensus.cc:2258] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:52.281764 26535 raft_consensus.cc:2258] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:52.283290 26537 tablet_replica.cc:330] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: stopping tablet replica
I20260811 09:14:52.283726 26537 raft_consensus.cc:2229] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:14:52.284473 26537 raft_consensus.cc:2258] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:14:52.285115 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.274231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.17.16.65:45759)
0811 09:14:52.274399 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:52.274405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:52.282659 (+ 8254us) server_negotiation.cc:407] Connection header received
0811 09:14:52.282869 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:52.282876 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:52.282983 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:52.283090 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:52.284278 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:52.284285 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:52.284289 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:52.284310 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:52.284350 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:52.284366 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:52.284465 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:52.284746 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 09:14:52.284812 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:14:52.286074 25995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:52.286406 26535 ts_tablet_manager.cc:1905] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:52.287155 26537 ts_tablet_manager.cc:1905] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:52.293359 26535 ts_tablet_manager.cc:1918] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:14:52.293565 26535 log.cc:1196] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/a28963da186f4381803ff713225acd93
I20260811 09:14:52.294117 26535 ts_tablet_manager.cc:1939] T a28963da186f4381803ff713225acd93 P 1e482c3dc9654ae6bb0305b47a4ffed1: Deleting consensus metadata
I20260811 09:14:52.294378 26537 ts_tablet_manager.cc:1918] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:14:52.294589 26537 log.cc:1196] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/a28963da186f4381803ff713225acd93
I20260811 09:14:52.295164 26537 ts_tablet_manager.cc:1939] T a28963da186f4381803ff713225acd93 P 538abc6dc8fa47658ccd54168f001195: Deleting consensus metadata
I20260811 09:14:52.296260 25657 catalog_manager.cc:4841] TS 1e482c3dc9654ae6bb0305b47a4ffed1 (127.17.16.66:46647): tablet a28963da186f4381803ff713225acd93 (table test-table [id=c7cab00ce6bd4f4f8996d2b32750180c]) successfully deleted
I20260811 09:14:52.296831 25661 catalog_manager.cc:4841] TS 538abc6dc8fa47658ccd54168f001195 (127.17.16.67:41395): tablet a28963da186f4381803ff713225acd93 (table test-table [id=c7cab00ce6bd4f4f8996d2b32750180c]) successfully deleted
I20260811 09:14:52.286191 26536 ts_tablet_manager.cc:1905] T a28963da186f4381803ff713225acd93 P 7234d10aacd34d44806e9a298072cc0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:14:52.382061 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 25785
I20260811 09:14:52.408411 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 26009
W20260811 09:14:52.408468 25662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.16.65:45759 (error 108)
I20260811 09:14:52.411032 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.410530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45759 (local address 127.0.0.1:41890)
0811 09:14:52.410710 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.410858 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 09:14:52.411284 25662 catalog_manager.cc:4584] TS 7234d10aacd34d44806e9a298072cc0a (127.17.16.65:45759): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a28963da186f4381803ff713225acd93: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45759: connect: Connection refused (error 111)
I20260811 09:14:52.411695 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.409209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34634 (local address 127.17.16.66:46647)
0811 09:14:52.409422 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:52.409429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:52.409673 (+ 244us) server_negotiation.cc:407] Connection header received
0811 09:14:52.409746 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:52.409751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:52.409864 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:52.409958 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:52.410943 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:52.410950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:52.410954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:52.410970 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:52.411001 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:52.411013 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:52.411126 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:52.411375 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 09:14:52.411440 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:14:52.412276 26222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:52.465160 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.464562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45759 (local address 127.0.0.1:41904)
0811 09:14:52.464808 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.464963 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 09:14:52.466351 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.465932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45759 (local address 127.0.0.1:41912)
0811 09:14:52.466102 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.466198 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 09:14:52.499205 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 26009
I20260811 09:14:52.523274 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 26240
I20260811 09:14:52.526340 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.524132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.17.16.67:41395)
0811 09:14:52.524312 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:52.524316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:52.524334 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:14:52.524548 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:52.524552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:52.524603 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:52.524686 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:52.525622 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:52.525626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:52.525629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:52.525646 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:52.525676 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:52.525690 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:52.525798 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:52.525938 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 09:14:52.526008 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:14:52.527012 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.526394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45759 (local address 127.0.0.1:41922)
0811 09:14:52.526742 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.526846 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214}
I20260811 09:14:52.527277 26454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:52.528034 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.527632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45759 (local address 127.0.0.1:41938)
0811 09:14:52.527823 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.527898 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 09:14:52.608502 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.608037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45759 (local address 127.0.0.1:41940)
0811 09:14:52.608272 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.608370 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 09:14:52.609561 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:52.609186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45759 (local address 127.0.0.1:41950)
0811 09:14:52.609375 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:52.609444 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 09:14:52.654198 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 26240
I20260811 09:14:52.683739 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 25629
I20260811 09:14:52.684938 25737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:52.790006 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 25629
2026-08-11T09:14:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2652 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 09:14:52.837749 17473 mini_oidc.cc:152] Starting JWKS server
I20260811 09:14:52.840637 17473 webserver.cc:460] Webserver started at https://127.0.0.1:33675/ using document root <none> and password file <none>
I20260811 09:14:52.840886 17473 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 09:14:52.841545 17473 webserver.cc:460] Webserver started at http://127.0.0.1:44183/ using document root <none> and password file <none>
2026-08-11T09:14:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:52Z Disabled control of system clock
I20260811 09:14:52.874349 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41469
--webserver_interface=127.17.16.126
--webserver_port=0
--builtin_ntp_servers=127.17.16.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41469
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33675/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/testchainca.cert with env {}
W20260811 09:14:53.193018 26554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:53.193279 26554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:53.193408 26554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:53.200321 26554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:53.200421 26554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:53.200495 26554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:53.200569 26554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:53.200629 26554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:14:53.200712 26554 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:14:53.200762 26554 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33675/jwks.json
I20260811 09:14:53.210256 26554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33675/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:53.211483 26554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:53.212762 26554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:53.223980 26566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:53.224020 26565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:53.223902 26570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:53.225091 26554 server_base.cc:1034] running on GCE node
I20260811 09:14:53.225832 26554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:53.227105 26554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:53.228307 26554 hybrid_clock.cc:648] HybridClock initialized: now 1786439693228260 us; error 38 us; skew 500 ppm
I20260811 09:14:53.228608 26554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:14:53.230825 26554 webserver.cc:460] Webserver started at http://127.17.16.126:41553/ using document root <none> and password file <none>
I20260811 09:14:53.231439 26554 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:53.231527 26554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:53.231781 26554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:53.234578 26554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "c0ab4ef940414d4dae079a56fb07d80b"
format_stamp: "Formatted at 2026-08-11 09:14:53 on dist-test-slave-ncp9"
server_key: "3b849049d2152a817e7530391af9dd84"
server_key_iv: "57ba90e0a330a7cf63407c0e29add3ea"
server_key_version: "encryptionkey@0"
I20260811 09:14:53.235374 26554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "c0ab4ef940414d4dae079a56fb07d80b"
format_stamp: "Formatted at 2026-08-11 09:14:53 on dist-test-slave-ncp9"
server_key: "3b849049d2152a817e7530391af9dd84"
server_key_iv: "57ba90e0a330a7cf63407c0e29add3ea"
server_key_version: "encryptionkey@0"
I20260811 09:14:53.240319 26554 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 09:14:53.243664 26579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:53.244964 26554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:14:53.245122 26554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "c0ab4ef940414d4dae079a56fb07d80b"
format_stamp: "Formatted at 2026-08-11 09:14:53 on dist-test-slave-ncp9"
server_key: "3b849049d2152a817e7530391af9dd84"
server_key_iv: "57ba90e0a330a7cf63407c0e29add3ea"
server_key_version: "encryptionkey@0"
I20260811 09:14:53.245332 26554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:53.273983 26554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:53.274756 26554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:53.275035 26554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:53.291759 26554 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:41469
I20260811 09:14:53.291791 26672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:41469 every 8 connection(s)
I20260811 09:14:53.293074 26554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:53.296664 26673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:53.301451 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 26554
I20260811 09:14:53.301673 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:53.302531 17473 external_mini_cluster.cc:1428] Setting key 11aeba63f83f00ab545f1a1330d3f7ae
I20260811 09:14:53.307735 26673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b: Bootstrap starting.
I20260811 09:14:53.311250 26673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:53.312333 26673 log.cc:824] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:53.315121 26673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b: No bootstrap required, opened a new log
I20260811 09:14:53.315814 26676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:53.304371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.17.16.126:41469)
0811 09:14:53.305357 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:53.305367 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:53.305414 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:14:53.307089 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:53.307111 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:53.307527 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:53.307847 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:53.308457 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:53.309782 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:53.311599 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:53.312251 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:53.312385 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:53.313723 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:53.313730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:53.313743 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:53.314235 (+ 492us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:53.314312 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:53.314333 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:53.314431 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:53.314655 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 09:14:53.314915 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":353,"threads_started":1}
I20260811 09:14:53.319432 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:14:53.323064 26673 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0ab4ef940414d4dae079a56fb07d80b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41469 } }
I20260811 09:14:53.323401 26673 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:53.323493 26673 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0ab4ef940414d4dae079a56fb07d80b, State: Initialized, Role: FOLLOWER
I20260811 09:14:53.324011 26673 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0ab4ef940414d4dae079a56fb07d80b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41469 } }
I20260811 09:14:53.324227 26673 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:53.324347 26673 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:53.324496 26673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:53.326646 26673 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0ab4ef940414d4dae079a56fb07d80b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41469 } }
I20260811 09:14:53.327111 26673 leader_election.cc:302] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0ab4ef940414d4dae079a56fb07d80b; no voters:
I20260811 09:14:53.327652 26673 leader_election.cc:288] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:53.327802 26679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:53.328848 26679 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [term 1 LEADER]: Becoming Leader. State: Replica: c0ab4ef940414d4dae079a56fb07d80b, State: Running, Role: LEADER
I20260811 09:14:53.329401 26679 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0ab4ef940414d4dae079a56fb07d80b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41469 } }
I20260811 09:14:53.329739 26673 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:53.332118 26680 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0ab4ef940414d4dae079a56fb07d80b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0ab4ef940414d4dae079a56fb07d80b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41469 } } }
I20260811 09:14:53.332788 26680 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:53.332396 26681 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0ab4ef940414d4dae079a56fb07d80b. Latest consensus state: current_term: 1 leader_uuid: "c0ab4ef940414d4dae079a56fb07d80b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0ab4ef940414d4dae079a56fb07d80b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41469 } } }
I20260811 09:14:53.334703 26681 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:53.337503 26689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:53.342759 26689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:53.351073 26689 catalog_manager.cc:1347] Generated new cluster ID: eea6d968b4a94962b21cf7c36c9a6b4d
I20260811 09:14:53.351183 26689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:53.375200 26689 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:53.377102 26689 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:53.387200 26689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0ab4ef940414d4dae079a56fb07d80b: Generated new TSK 0
I20260811 09:14:53.388180 26689 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 09:14:53.439893 26715 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 09:14:53.440150 26715 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 09:14:53.441000 26676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:53.411552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.17.16.126:41469)
0811 09:14:53.411786 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:53.411794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:53.412252 (+ 458us) server_negotiation.cc:407] Connection header received
0811 09:14:53.412625 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:53.412629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:53.412695 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:53.412812 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:14:53.413460 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:53.414228 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:53.416264 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:53.416741 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:53.416828 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:53.417461 (+ 633us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:14:53.440417 (+ 22956us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 09:14:53.440733 (+ 316us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52240: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 09:14:53.441138 26676 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52240: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 09:14:53.443333 26714 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:14:53.411424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41469 (local address 127.0.0.1:52240)
0811 09:14:53.412145 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:53.412166 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:53.412353 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:53.412855 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:53.412861 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:53.412873 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:14:53.413308 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:53.413321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:53.414359 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:53.414365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:53.416110 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:53.416127 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:53.416941 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:53.416949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:53.417139 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:53.417214 (+ 75us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:14:53.440732 (+ 23518us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 09:14:53.441159 (+ 427us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.16.126:41469: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":365,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.16.126:41469: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 09:14:53.449863 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 26554
I20260811 09:14:53.451370 26667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:53.547760 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 26554
2026-08-11T09:14:53Z chronyd exiting
I20260811 09:14:53.602213 17473 test_util.cc:175] -----------------------------------------------
I20260811 09:14:53.602340 17473 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439649204149-17473-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (769 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 09:14:53.606333 17473 mini_oidc.cc:152] Starting JWKS server
I20260811 09:14:53.608119 17473 webserver.cc:460] Webserver started at https://127.0.0.1:34071/ using document root <none> and password file <none>
I20260811 09:14:53.608346 17473 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 09:14:53.608954 17473 webserver.cc:460] Webserver started at http://127.0.0.1:32949/ using document root <none> and password file <none>
2026-08-11T09:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:53Z Disabled control of system clock
I20260811 09:14:53.641812 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:35621
--webserver_interface=127.17.16.126
--webserver_port=0
--builtin_ntp_servers=127.17.16.84:36709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:35621
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34071/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/testchainca.cert with env {}
W20260811 09:14:53.964681 26729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:53.965014 26729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:53.965161 26729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:53.972349 26729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:53.972468 26729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:53.972536 26729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:53.972617 26729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:53.972673 26729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:14:53.972759 26729 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:14:53.972808 26729 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34071/jwks.json
I20260811 09:14:53.982980 26729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:36709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:35621
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:35621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34071/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:53.984228 26729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:53.985563 26729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:53.996654 26739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:53.996726 26745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:53.996701 26740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:53.997418 26729 server_base.cc:1034] running on GCE node
I20260811 09:14:53.998126 26729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:53.999405 26729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:54.000603 26729 hybrid_clock.cc:648] HybridClock initialized: now 1786439694000555 us; error 37 us; skew 500 ppm
I20260811 09:14:54.000959 26729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:14:54.006224 26729 webserver.cc:460] Webserver started at http://127.17.16.126:37113/ using document root <none> and password file <none>
I20260811 09:14:54.006914 26729 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:54.007023 26729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:54.007352 26729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:54.010347 26729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "e48b624425614c8ea1c8ff9024cd8d0b"
format_stamp: "Formatted at 2026-08-11 09:14:54 on dist-test-slave-ncp9"
server_key: "32b139f5ad0dbb888e0042a02e5de07f"
server_key_iv: "2c630a6510938bce79a9c8cdbb0db260"
server_key_version: "encryptionkey@0"
I20260811 09:14:54.011125 26729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "e48b624425614c8ea1c8ff9024cd8d0b"
format_stamp: "Formatted at 2026-08-11 09:14:54 on dist-test-slave-ncp9"
server_key: "32b139f5ad0dbb888e0042a02e5de07f"
server_key_iv: "2c630a6510938bce79a9c8cdbb0db260"
server_key_version: "encryptionkey@0"
I20260811 09:14:54.016098 26729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:14:54.019393 26758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:54.020720 26729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:14:54.020872 26729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "e48b624425614c8ea1c8ff9024cd8d0b"
format_stamp: "Formatted at 2026-08-11 09:14:54 on dist-test-slave-ncp9"
server_key: "32b139f5ad0dbb888e0042a02e5de07f"
server_key_iv: "2c630a6510938bce79a9c8cdbb0db260"
server_key_version: "encryptionkey@0"
I20260811 09:14:54.021117 26729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:54.040262 26729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:54.041221 26729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:54.041545 26729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:54.059530 26729 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:35621
I20260811 09:14:54.059566 26838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:35621 every 8 connection(s)
I20260811 09:14:54.060842 26729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:54.064556 26839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:54.071367 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 26729
I20260811 09:14:54.071611 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:54.072376 17473 external_mini_cluster.cc:1428] Setting key 189b13df872791a2a42a688a0477ca55
I20260811 09:14:54.075248 26839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b: Bootstrap starting.
I20260811 09:14:54.078574 26839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:54.079588 26839 log.cc:824] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:54.082504 26839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b: No bootstrap required, opened a new log
I20260811 09:14:54.085343 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:54.074230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.17.16.126:35621)
0811 09:14:54.075134 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:54.075147 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:54.075188 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:14:54.076871 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:54.076923 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:54.077369 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:54.077675 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:54.078240 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.079572 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.081068 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.081727 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.081848 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:54.083127 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:54.083134 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:54.083150 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:54.083649 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:54.083723 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:54.083743 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:54.083824 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:54.084057 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 09:14:54.084359 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":316,"threads_started":1}
I20260811 09:14:54.089156 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:14:54.090734 26839 raft_consensus.cc:348] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48b624425614c8ea1c8ff9024cd8d0b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35621 } }
I20260811 09:14:54.091095 26839 raft_consensus.cc:374] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:54.091197 26839 raft_consensus.cc:729] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48b624425614c8ea1c8ff9024cd8d0b, State: Initialized, Role: FOLLOWER
I20260811 09:14:54.091826 26839 consensus_queue.cc:260] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48b624425614c8ea1c8ff9024cd8d0b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35621 } }
I20260811 09:14:54.092113 26839 raft_consensus.cc:388] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:54.092248 26839 raft_consensus.cc:482] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:54.092401 26839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:54.094695 26839 raft_consensus.cc:504] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48b624425614c8ea1c8ff9024cd8d0b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35621 } }
I20260811 09:14:54.095160 26839 leader_election.cc:302] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e48b624425614c8ea1c8ff9024cd8d0b; no voters:
I20260811 09:14:54.095690 26839 leader_election.cc:288] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:54.095831 26847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:54.096513 26847 raft_consensus.cc:686] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [term 1 LEADER]: Becoming Leader. State: Replica: e48b624425614c8ea1c8ff9024cd8d0b, State: Running, Role: LEADER
I20260811 09:14:54.097100 26847 consensus_queue.cc:237] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48b624425614c8ea1c8ff9024cd8d0b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35621 } }
I20260811 09:14:54.097643 26839 sys_catalog.cc:564] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:54.100134 26853 sys_catalog.cc:455] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e48b624425614c8ea1c8ff9024cd8d0b. Latest consensus state: current_term: 1 leader_uuid: "e48b624425614c8ea1c8ff9024cd8d0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48b624425614c8ea1c8ff9024cd8d0b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35621 } } }
I20260811 09:14:54.100569 26853 sys_catalog.cc:458] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:54.100154 26849 sys_catalog.cc:455] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e48b624425614c8ea1c8ff9024cd8d0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e48b624425614c8ea1c8ff9024cd8d0b" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35621 } } }
I20260811 09:14:54.101768 26849 sys_catalog.cc:458] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:54.101864 26864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:54.106140 26864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:54.114807 26864 catalog_manager.cc:1347] Generated new cluster ID: 9a8aa6df6de84c71a96fe5dccb4016ce
I20260811 09:14:54.114946 26864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:54.139621 26864 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:54.140883 26864 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:54.153118 26864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e48b624425614c8ea1c8ff9024cd8d0b: Generated new TSK 0
I20260811 09:14:54.153868 26864 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 09:14:54.191402 26898 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 09:14:54.191628 26898 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 09:14:54.192581 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:54.165191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.17.16.126:35621)
0811 09:14:54.165381 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:54.165389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:54.165915 (+ 526us) server_negotiation.cc:407] Connection header received
0811 09:14:54.166155 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:54.166158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:54.166232 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:54.166359 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:14:54.166898 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.167617 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.169585 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.170014 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.170099 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:54.170643 (+ 544us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:14:54.192074 (+ 21431us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34071/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 09:14:54.192385 (+ 311us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49620: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34071/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 09:14:54.192705 26844 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49620: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34071/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 09:14:54.193034 26895 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:14:54.165174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35621 (local address 127.0.0.1:49620)
0811 09:14:54.165804 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:54.165829 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:54.165983 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:54.166385 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:54.166391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:54.166404 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:14:54.166730 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:54.166743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.167746 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.167751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:54.169450 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:54.169466 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.170164 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.170170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:54.170363 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:54.170409 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:14:54.192314 (+ 21905us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34071/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 09:14:54.192597 (+ 283us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.16.126:35621: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34071/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":310,"threads_started":1}
I20260811 09:14:54.194463 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 26729
I20260811 09:14:54.195510 26830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:54.287544 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 26729
2026-08-11T09:14:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (741 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 09:14:54.347342 17473 mini_oidc.cc:152] Starting JWKS server
I20260811 09:14:54.349257 17473 webserver.cc:460] Webserver started at https://127.0.0.1:39583/ using document root <none> and password file <none>
I20260811 09:14:54.349476 17473 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 09:14:54.350085 17473 webserver.cc:460] Webserver started at http://127.0.0.1:36755/ using document root <none> and password file <none>
2026-08-11T09:14:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:54Z Disabled control of system clock
I20260811 09:14:54.383052 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:42615
--webserver_interface=127.17.16.126
--webserver_port=0
--builtin_ntp_servers=127.17.16.84:37149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:42615
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39583/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 09:14:54.704062 26911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:54.704350 26911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:54.704481 26911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:54.711589 26911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:54.711706 26911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:54.711767 26911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:54.711853 26911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:54.711908 26911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:14:54.711994 26911 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:14:54.712051 26911 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39583/jwks.json
I20260811 09:14:54.721125 26911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:42615
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:42615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39583/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:54.722424 26911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:54.724233 26911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:54.735667 26919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:54.735705 26920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:54.735728 26922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:54.736488 26911 server_base.cc:1034] running on GCE node
I20260811 09:14:54.737326 26911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:54.738751 26911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:54.739985 26911 hybrid_clock.cc:648] HybridClock initialized: now 1786439694739928 us; error 42 us; skew 500 ppm
I20260811 09:14:54.740345 26911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:14:54.742621 26911 webserver.cc:460] Webserver started at http://127.17.16.126:43925/ using document root <none> and password file <none>
I20260811 09:14:54.743288 26911 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:54.743391 26911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:54.743701 26911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:54.746531 26911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "e609fbac2c9d4b81bdbb529aa007d9d0"
format_stamp: "Formatted at 2026-08-11 09:14:54 on dist-test-slave-ncp9"
server_key: "fbd2dc95b7c747afc92c24fbceaeb03b"
server_key_iv: "45c7e9163ec72ffc4145e494ba047119"
server_key_version: "encryptionkey@0"
I20260811 09:14:54.747301 26911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "e609fbac2c9d4b81bdbb529aa007d9d0"
format_stamp: "Formatted at 2026-08-11 09:14:54 on dist-test-slave-ncp9"
server_key: "fbd2dc95b7c747afc92c24fbceaeb03b"
server_key_iv: "45c7e9163ec72ffc4145e494ba047119"
server_key_version: "encryptionkey@0"
I20260811 09:14:54.752122 26911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 09:14:54.755239 26934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:54.756449 26911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 09:14:54.756603 26911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "e609fbac2c9d4b81bdbb529aa007d9d0"
format_stamp: "Formatted at 2026-08-11 09:14:54 on dist-test-slave-ncp9"
server_key: "fbd2dc95b7c747afc92c24fbceaeb03b"
server_key_iv: "45c7e9163ec72ffc4145e494ba047119"
server_key_version: "encryptionkey@0"
I20260811 09:14:54.756788 26911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:54.793679 26911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:54.794512 26911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:54.794811 26911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:54.812255 26911 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:42615
I20260811 09:14:54.812294 27022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:42615 every 8 connection(s)
I20260811 09:14:54.813648 26911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:54.817318 27024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:54.822448 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 26911
I20260811 09:14:54.822649 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:54.823585 17473 external_mini_cluster.cc:1428] Setting key d1f8f6bf9ded6d85e3060ed1e4849a11
I20260811 09:14:54.828660 27024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0: Bootstrap starting.
I20260811 09:14:54.832108 27024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:54.833325 27024 log.cc:824] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:54.836365 27024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0: No bootstrap required, opened a new log
I20260811 09:14:54.837167 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:54.825575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.17.16.126:42615)
0811 09:14:54.826673 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:54.826684 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:54.826727 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:14:54.828366 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:54.828388 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:54.828833 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:54.829207 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:54.829771 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.831104 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.832874 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.833629 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.833796 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:54.834999 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:54.835006 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:54.835026 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:54.835511 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:54.835586 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:54.835609 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:54.835689 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:54.835925 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 09:14:54.836222 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":422,"threads_started":1}
I20260811 09:14:54.840718 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:14:54.844911 27024 raft_consensus.cc:348] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e609fbac2c9d4b81bdbb529aa007d9d0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42615 } }
I20260811 09:14:54.845243 27024 raft_consensus.cc:374] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:54.845314 27024 raft_consensus.cc:729] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e609fbac2c9d4b81bdbb529aa007d9d0, State: Initialized, Role: FOLLOWER
I20260811 09:14:54.845813 27024 consensus_queue.cc:260] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e609fbac2c9d4b81bdbb529aa007d9d0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42615 } }
I20260811 09:14:54.846020 27024 raft_consensus.cc:388] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:54.846120 27024 raft_consensus.cc:482] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:54.846251 27024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:54.848318 27024 raft_consensus.cc:504] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e609fbac2c9d4b81bdbb529aa007d9d0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42615 } }
I20260811 09:14:54.848865 27024 leader_election.cc:302] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e609fbac2c9d4b81bdbb529aa007d9d0; no voters:
I20260811 09:14:54.849417 27024 leader_election.cc:288] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:54.849517 27033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:54.850200 27033 raft_consensus.cc:686] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [term 1 LEADER]: Becoming Leader. State: Replica: e609fbac2c9d4b81bdbb529aa007d9d0, State: Running, Role: LEADER
I20260811 09:14:54.850700 27033 consensus_queue.cc:237] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e609fbac2c9d4b81bdbb529aa007d9d0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42615 } }
I20260811 09:14:54.851402 27024 sys_catalog.cc:564] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:54.853312 27034 sys_catalog.cc:455] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e609fbac2c9d4b81bdbb529aa007d9d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e609fbac2c9d4b81bdbb529aa007d9d0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42615 } } }
I20260811 09:14:54.853698 27034 sys_catalog.cc:458] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:54.854907 27038 sys_catalog.cc:455] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e609fbac2c9d4b81bdbb529aa007d9d0. Latest consensus state: current_term: 1 leader_uuid: "e609fbac2c9d4b81bdbb529aa007d9d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e609fbac2c9d4b81bdbb529aa007d9d0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42615 } } }
I20260811 09:14:54.855233 27038 sys_catalog.cc:458] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:54.857770 27045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:54.862790 27045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:54.870782 27045 catalog_manager.cc:1347] Generated new cluster ID: 5fa22bb107bc4980a5c508b8a88a3610
I20260811 09:14:54.870893 27045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:54.889714 27045 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:54.891516 27045 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:54.904445 27045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e609fbac2c9d4b81bdbb529aa007d9d0: Generated new TSK 0
I20260811 09:14:54.905263 27045 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:14:54.934355 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:54.925843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38838 (local address 127.17.16.126:42615)
0811 09:14:54.926052 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:54.926058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:54.926286 (+ 228us) server_negotiation.cc:407] Connection header received
0811 09:14:54.926508 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:54.926514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:54.926611 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:54.926786 (+ 175us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:14:54.927199 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.928374 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.929865 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.930598 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.930737 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:54.931294 (+ 557us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:14:54.933690 (+ 2396us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39583/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 09:14:54.934046 (+ 356us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38838: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39583/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 09:14:54.934504 27030 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38838: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39583/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 09:14:54.935990 27069 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:14:54.925637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:42615 (local address 127.0.0.1:38838)
0811 09:14:54.926198 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:54.926214 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:54.926338 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:54.926782 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:54.926785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:54.926795 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:14:54.927060 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:54.927068 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.928502 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.928505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:54.929702 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:54.929720 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:54.930812 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:54.930817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:54.930956 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:54.931014 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:14:54.933918 (+ 2904us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39583/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 09:14:54.935544 (+ 1626us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.16.126:42615: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39583/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":1318,"thread_start_us":301,"threads_started":1}
I20260811 09:14:54.937491 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 26911
I20260811 09:14:54.938758 27015 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 09:14:54.945405 27070 webserver.cc:533] Webserver: sslize failed:
E20260811 09:14:54.945488 27070 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 09:14:55.029826 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 26911
2026-08-11T09:14:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (744 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T09:14:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:55Z Disabled control of system clock
I20260811 09:14:55.124305 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:38829
--webserver_interface=127.17.16.126
--webserver_port=0
--builtin_ntp_servers=127.17.16.84:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:38829
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/testchainca.cert with env {}
W20260811 09:14:55.445264 27076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:55.445644 27076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:55.445791 27076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:55.453685 27076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:55.453792 27076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:55.453852 27076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:55.453925 27076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:55.453984 27076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:14:55.454066 27076 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:14:55.454114 27076 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 09:14:55.464107 27076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:38829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:38829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:55.465394 27076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:55.466670 27076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:55.477901 27084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:55.477943 27093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:55.477975 27085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:55.478801 27076 server_base.cc:1034] running on GCE node
I20260811 09:14:55.479477 27076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:55.480724 27076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:55.481920 27076 hybrid_clock.cc:648] HybridClock initialized: now 1786439695481863 us; error 44 us; skew 500 ppm
I20260811 09:14:55.482259 27076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:14:55.484453 27076 webserver.cc:460] Webserver started at http://127.17.16.126:45165/ using document root <none> and password file <none>
I20260811 09:14:55.485199 27076 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:55.485311 27076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:55.485616 27076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:55.488440 27076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "36c771b88d9c4465aaa0fbf8fd9ea415"
format_stamp: "Formatted at 2026-08-11 09:14:55 on dist-test-slave-ncp9"
server_key: "0ff272b4aa94f2994b776a3f9f52e3cf"
server_key_iv: "8a1ad152f8fff675aa03927bd1dd5718"
server_key_version: "encryptionkey@0"
I20260811 09:14:55.489477 27076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "36c771b88d9c4465aaa0fbf8fd9ea415"
format_stamp: "Formatted at 2026-08-11 09:14:55 on dist-test-slave-ncp9"
server_key: "0ff272b4aa94f2994b776a3f9f52e3cf"
server_key_iv: "8a1ad152f8fff675aa03927bd1dd5718"
server_key_version: "encryptionkey@0"
I20260811 09:14:55.495065 27076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:14:55.498459 27100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:55.499752 27076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:14:55.499918 27076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "36c771b88d9c4465aaa0fbf8fd9ea415"
format_stamp: "Formatted at 2026-08-11 09:14:55 on dist-test-slave-ncp9"
server_key: "0ff272b4aa94f2994b776a3f9f52e3cf"
server_key_iv: "8a1ad152f8fff675aa03927bd1dd5718"
server_key_version: "encryptionkey@0"
I20260811 09:14:55.500123 27076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:55.519750 27076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:55.520490 27076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:55.520752 27076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:55.538026 27076 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:38829
I20260811 09:14:55.538045 27181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:38829 every 8 connection(s)
I20260811 09:14:55.539294 27076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:55.542876 27183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:55.544265 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 27076
I20260811 09:14:55.544497 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:55.545354 17473 external_mini_cluster.cc:1428] Setting key 25d8589e80bed8b3615d4015b578c9e5
I20260811 09:14:55.556291 27183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415: Bootstrap starting.
I20260811 09:14:55.558321 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:55.547182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.17.16.126:38829)
0811 09:14:55.548150 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:55.548161 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:55.548201 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:14:55.549768 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:55.549794 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:55.550283 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:55.550585 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:55.551176 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:55.552431 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:55.554209 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:55.554837 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:55.554959 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:55.556170 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:55.556177 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:55.556190 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:14:55.556715 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:55.556789 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:14:55.556810 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:55.556920 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:55.557137 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 09:14:55.557412 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":360,"threads_started":1}
I20260811 09:14:55.559722 27183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:55.560809 27183 log.cc:824] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415: Log is configured to *not* fsync() on all Append() calls
I20260811 09:14:55.561949 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:14:55.563472 27183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415: No bootstrap required, opened a new log
I20260811 09:14:55.571094 27183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c771b88d9c4465aaa0fbf8fd9ea415" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38829 } }
I20260811 09:14:55.571524 27183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:55.571633 27183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c771b88d9c4465aaa0fbf8fd9ea415, State: Initialized, Role: FOLLOWER
I20260811 09:14:55.572186 27183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c771b88d9c4465aaa0fbf8fd9ea415" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38829 } }
I20260811 09:14:55.572391 27183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:55.572520 27183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:55.572677 27183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:55.574795 27183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c771b88d9c4465aaa0fbf8fd9ea415" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38829 } }
I20260811 09:14:55.575273 27183 leader_election.cc:302] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36c771b88d9c4465aaa0fbf8fd9ea415; no voters:
I20260811 09:14:55.575830 27183 leader_election.cc:288] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:55.575976 27189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:55.576598 27189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [term 1 LEADER]: Becoming Leader. State: Replica: 36c771b88d9c4465aaa0fbf8fd9ea415, State: Running, Role: LEADER
I20260811 09:14:55.577147 27189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c771b88d9c4465aaa0fbf8fd9ea415" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38829 } }
I20260811 09:14:55.577816 27183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:55.579769 27192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36c771b88d9c4465aaa0fbf8fd9ea415. Latest consensus state: current_term: 1 leader_uuid: "36c771b88d9c4465aaa0fbf8fd9ea415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c771b88d9c4465aaa0fbf8fd9ea415" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38829 } } }
I20260811 09:14:55.580127 27192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:55.581800 27190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36c771b88d9c4465aaa0fbf8fd9ea415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c771b88d9c4465aaa0fbf8fd9ea415" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38829 } } }
I20260811 09:14:55.581988 27200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:55.582057 27190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:55.587076 27200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:55.595345 27200 catalog_manager.cc:1347] Generated new cluster ID: 930f43adfd134f00ae4cf328d2ab22b0
I20260811 09:14:55.595448 27200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:55.609118 27200 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:55.610839 27200 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:55.621454 27200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36c771b88d9c4465aaa0fbf8fd9ea415: Generated new TSK 0
I20260811 09:14:55.622511 27200 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:14:56.205528 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:55.639786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55240 (local address 127.17.16.126:38829)
0811 09:14:55.639971 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:55.639978 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:55.640313 (+ 335us) server_negotiation.cc:407] Connection header received
0811 09:14:55.640564 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:55.640570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:55.640660 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:55.640824 (+ 164us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:14:55.641275 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:55.642434 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:55.643930 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:55.644634 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:55.644756 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:55.645283 (+ 527us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:14:56.204955 (+559672us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 09:14:56.205307 (+ 352us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55240: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 09:14:56.205660 27187 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55240: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 09:14:56.205857 27238 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:14:55.639616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:38829 (local address 127.0.0.1:55240)
0811 09:14:55.640215 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:55.640231 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:55.640354 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:55.640819 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:55.640822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:55.640831 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:14:55.641122 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:55.641131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:55.642599 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:55.642604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:55.643786 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:55.643800 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:55.644801 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:55.644806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:55.644971 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:55.645007 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:14:56.205243 (+560236us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 09:14:56.205498 (+ 255us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.16.126:38829: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":298,"threads_started":1}
I20260811 09:14:56.207499 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 27076
I20260811 09:14:56.208527 27173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:56.309602 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 27076
2026-08-11T09:14:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1259 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (422 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (411 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:14:57 dist-test-slave-ncp9 krb5kdc[27265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:14:57 dist-test-slave-ncp9 krb5kdc[27265](info): set up 2 sockets
Aug 11 09:14:57 dist-test-slave-ncp9 krb5kdc[27265](info): commencing operation
krb5kdc: starting...
W20260811 09:14:58.357894 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:14:58 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:14:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:14:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:14:58.584509 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46867
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:46867
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:58.911063 27295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:58.911360 27295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:58.911501 27295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:58.918751 27295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:14:58.919010 27295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:58.919113 27295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:58.919209 27295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:14:58.919278 27295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:14:58.929277 27295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:46867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:58.930632 27295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:58.932093 27295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:58.943657 27303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:58.943708 27304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:58.943643 27307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:58.944850 27295 server_base.cc:1034] running on GCE node
I20260811 09:14:58.945641 27295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:58.946933 27295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:58.948130 27295 hybrid_clock.cc:648] HybridClock initialized: now 1786439698948088 us; error 30 us; skew 500 ppm
I20260811 09:14:58.948458 27295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:58 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439698, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:58.951495 27295 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:14:58.952929 27295 webserver.cc:460] Webserver started at http://127.17.16.126:42167/ using document root <none> and password file <none>
I20260811 09:14:58.953576 27295 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:58.953707 27295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:58.954007 27295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:58.956933 27295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "a53d015700d143139957ee3acc843b40"
format_stamp: "Formatted at 2026-08-11 09:14:58 on dist-test-slave-ncp9"
server_key: "1d9ebcbb61d89190207b4fae845b8903"
server_key_iv: "1734e1e9a1bee265e83ed44f66acb992"
server_key_version: "encryptionkey@0"
I20260811 09:14:58.957713 27295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "a53d015700d143139957ee3acc843b40"
format_stamp: "Formatted at 2026-08-11 09:14:58 on dist-test-slave-ncp9"
server_key: "1d9ebcbb61d89190207b4fae845b8903"
server_key_iv: "1734e1e9a1bee265e83ed44f66acb992"
server_key_version: "encryptionkey@0"
I20260811 09:14:58.962774 27295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:14:58.966171 27317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:58.967419 27295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 09:14:58.967574 27295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "a53d015700d143139957ee3acc843b40"
format_stamp: "Formatted at 2026-08-11 09:14:58 on dist-test-slave-ncp9"
server_key: "1d9ebcbb61d89190207b4fae845b8903"
server_key_iv: "1734e1e9a1bee265e83ed44f66acb992"
server_key_version: "encryptionkey@0"
I20260811 09:14:58.967763 27295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:58.990200 27295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:58.994320 27295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:58.994578 27295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:59.012147 27295 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:46867
I20260811 09:14:59.012181 27411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:46867 every 8 connection(s)
I20260811 09:14:59.013468 27295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:14:59.017043 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 27295
I20260811 09:14:59.017290 27413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:14:59.017298 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:14:59.018163 17473 external_mini_cluster.cc:1428] Setting key 37b496914bf2bbba0a516584ae71a329
I20260811 09:14:59.030663 27413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40: Bootstrap starting.
I20260811 09:14:59.033440 27413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40: Neither blocks nor log segments found. Creating new log.
I20260811 09:14:59.034621 27413 log.cc:824] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:14:59 dist-test-slave-ncp9 krb5kdc[27265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:59.037297 27413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40: No bootstrap required, opened a new log
I20260811 09:14:59.044242 27416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:59.020211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41206 (local address 127.17.16.126:46867)
0811 09:14:59.021029 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:59.021041 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:59.021080 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:14:59.023354 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:59.023377 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:59.023742 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:59.024051 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:59.025766 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:59.027026 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:59.028758 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:59.029457 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:59.029586 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:59.037305 (+ 7719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:59.037315 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:59.037329 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:59.037369 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:59.040357 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:59.041012 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:59.041019 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:59.041024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:59.041203 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:59.041603 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:59.041606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:59.041608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:59.042124 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:59.042441 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:59.042872 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 09:14:59.043262 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":378,"threads_started":1}
I20260811 09:14:59.045054 27413 raft_consensus.cc:348] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a53d015700d143139957ee3acc843b40" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46867 } }
I20260811 09:14:59.045396 27413 raft_consensus.cc:374] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:14:59.045508 27413 raft_consensus.cc:729] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53d015700d143139957ee3acc843b40, State: Initialized, Role: FOLLOWER
I20260811 09:14:59.046039 27413 consensus_queue.cc:260] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a53d015700d143139957ee3acc843b40" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46867 } }
I20260811 09:14:59.046263 27413 raft_consensus.cc:388] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:14:59.046391 27413 raft_consensus.cc:482] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:14:59.046561 27413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:14:59.048707 27413 raft_consensus.cc:504] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a53d015700d143139957ee3acc843b40" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46867 } }
I20260811 09:14:59.049176 27413 leader_election.cc:302] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a53d015700d143139957ee3acc843b40; no voters:
I20260811 09:14:59.049737 27413 leader_election.cc:288] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:14:59.049850 27419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:14:59.051059 27419 raft_consensus.cc:686] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [term 1 LEADER]: Becoming Leader. State: Replica: a53d015700d143139957ee3acc843b40, State: Running, Role: LEADER
I20260811 09:14:59.051656 27419 consensus_queue.cc:237] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a53d015700d143139957ee3acc843b40" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46867 } }
I20260811 09:14:59.052230 27413 sys_catalog.cc:564] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:14:59.054826 27422 sys_catalog.cc:455] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a53d015700d143139957ee3acc843b40. Latest consensus state: current_term: 1 leader_uuid: "a53d015700d143139957ee3acc843b40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a53d015700d143139957ee3acc843b40" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46867 } } }
I20260811 09:14:59.054947 27420 sys_catalog.cc:455] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a53d015700d143139957ee3acc843b40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a53d015700d143139957ee3acc843b40" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46867 } } }
I20260811 09:14:59.055222 27422 sys_catalog.cc:458] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:59.055258 27420 sys_catalog.cc:458] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40 [sys.catalog]: This master's current role is: LEADER
I20260811 09:14:59.057603 27430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:14:59.062728 27430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:14:59.071009 27430 catalog_manager.cc:1347] Generated new cluster ID: 1905348406124320a8bd933ed894f8a1
I20260811 09:14:59.071162 27430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:14:59.078672 27430 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:14:59.079824 27430 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:14:59.086724 27430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a53d015700d143139957ee3acc843b40: Generated new TSK 0
I20260811 09:14:59.087432 27430 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:14:59.156121 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46867
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:59.483180 27451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:14:59.483458 27451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:14:59.483552 27451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:14:59.490788 27451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:14:59.490908 27451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:14:59.491027 27451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:14:59.501143 27451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:14:59.502408 27451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:14:59.503775 27451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:14:59.516284 27463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:59.516435 27465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:14:59.516265 27459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:14:59.517518 27451 server_base.cc:1034] running on GCE node
I20260811 09:14:59.518102 27451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:14:59.518962 27451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:14:59.520164 27451 hybrid_clock.cc:648] HybridClock initialized: now 1786439699520113 us; error 44 us; skew 500 ppm
I20260811 09:14:59.520452 27451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:14:59 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439699, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:14:59.523504 27451 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:14:59.524955 27451 webserver.cc:460] Webserver started at http://127.17.16.65:39139/ using document root <none> and password file <none>
I20260811 09:14:59.525621 27451 fs_manager.cc:359] Metadata directory not provided
I20260811 09:14:59.525741 27451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:14:59.526074 27451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:14:59.529068 27451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "758c99cc59d64b4c9ed50f74dea0c7d4"
format_stamp: "Formatted at 2026-08-11 09:14:59 on dist-test-slave-ncp9"
server_key: "0872a77bca224148617c07b4c80eb149"
server_key_iv: "5e56e0a88b7951a706b2883bd732bd1e"
server_key_version: "encryptionkey@0"
I20260811 09:14:59.529939 27451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "758c99cc59d64b4c9ed50f74dea0c7d4"
format_stamp: "Formatted at 2026-08-11 09:14:59 on dist-test-slave-ncp9"
server_key: "0872a77bca224148617c07b4c80eb149"
server_key_iv: "5e56e0a88b7951a706b2883bd732bd1e"
server_key_version: "encryptionkey@0"
I20260811 09:14:59.535408 27451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 09:14:59.539531 27475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:59.541111 27451 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:14:59.541286 27451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "758c99cc59d64b4c9ed50f74dea0c7d4"
format_stamp: "Formatted at 2026-08-11 09:14:59 on dist-test-slave-ncp9"
server_key: "0872a77bca224148617c07b4c80eb149"
server_key_iv: "5e56e0a88b7951a706b2883bd732bd1e"
server_key_version: "encryptionkey@0"
I20260811 09:14:59.541594 27451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:14:59.560786 27451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:14:59.564738 27451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:14:59.565032 27451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:14:59.565980 27451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:14:59.567302 27451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:14:59.567376 27451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:59.567469 27451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:14:59.567528 27451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:14:59.597718 27451 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:39023
I20260811 09:14:59.597756 27657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:39023 every 8 connection(s)
I20260811 09:14:59.599010 27451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:14:59.608883 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 27451
I20260811 09:14:59.609151 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:14:59.609889 17473 external_mini_cluster.cc:1428] Setting key 22588d51e0086b624b562d9ee2249b63
Aug 11 09:14:59 dist-test-slave-ncp9 krb5kdc[27265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439699, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:14:59.618594 27664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:59.601443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:42897 (local address 127.17.16.126:46867)
0811 09:14:59.601937 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 09:14:59.601942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:14:59.602840 (+ 898us) server_negotiation.cc:407] Connection header received
0811 09:14:59.604684 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:14:59.604688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:14:59.604757 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:14:59.604852 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:14:59.606262 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:59.606995 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:59.608518 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:14:59.609176 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:14:59.609328 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:59.612942 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:14:59.612950 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:14:59.612956 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:14:59.612995 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:14:59.615434 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:59.616170 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:59.616177 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:59.616180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:59.616314 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:14:59.616754 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:14:59.616760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:14:59.616762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:14:59.617089 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:14:59.617235 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 09:14:59.617958 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 09:14:59.618210 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":235,"threads_started":1}
I20260811 09:14:59.619580 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:14:59.601948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46867 (local address 127.17.16.65:42897)
0811 09:14:59.602685 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 09:14:59.602727 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:14:59.604432 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:14:59.605087 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:14:59.605096 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:14:59.605575 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:14:59.606123 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:59.606141 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:59.607125 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:59.607129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:59.608346 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:14:59.608381 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:14:59.609397 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:14:59.609402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:14:59.609610 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:14:59.610262 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:14:59.610284 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:14:59.612691 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:14:59.615574 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:59.615583 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:59.615601 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:59.616043 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:59.616425 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:14:59.616430 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:14:59.616434 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:14:59.616646 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:14:59.617215 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:14:59.617223 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:14:59.617578 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 09:14:59.617844 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 09:14:59.618408 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":299,"threads_started":1}
I20260811 09:14:59.621174 27658 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46867
I20260811 09:14:59.621503 27658 heartbeater.cc:464] Registering TS with master...
I20260811 09:14:59.622411 27658 heartbeater.cc:511] Master 127.17.16.126:46867 requested a full tablet report, sending...
I20260811 09:14:59.624567 27343 ts_manager.cc:194] Registered new tserver with Master: 758c99cc59d64b4c9ed50f74dea0c7d4 (127.17.16.65:39023)
I20260811 09:14:59.626049 27343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:42897
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:14:59.665094 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46867
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:14:59.999785 27672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:00.000078 27672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:00.000231 27672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:00.008255 27672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:00.008375 27672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:00.008566 27672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:15:00.018767 27672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:00.020020 27672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:00.021364 27672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:00.034130 27681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:00.034245 27687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:00.034350 27684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:00.035283 27672 server_base.cc:1034] running on GCE node
I20260811 09:15:00.035923 27672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:00.036871 27672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:00.038172 27672 hybrid_clock.cc:648] HybridClock initialized: now 1786439700038107 us; error 61 us; skew 500 ppm
I20260811 09:15:00.038539 27672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:00.041630 27672 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:15:00.043045 27672 webserver.cc:460] Webserver started at http://127.17.16.66:42287/ using document root <none> and password file <none>
I20260811 09:15:00.043718 27672 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:00.043839 27672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:00.044137 27672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:00.047116 27672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "dd7fd13bc5d647cfa1018ea55c83cbbe"
format_stamp: "Formatted at 2026-08-11 09:15:00 on dist-test-slave-ncp9"
server_key: "ce9a2462da3f6cbbacef5422ce804be0"
server_key_iv: "dfaeffd40838fbd53f89e805b7dd269c"
server_key_version: "encryptionkey@0"
I20260811 09:15:00.047935 27672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "dd7fd13bc5d647cfa1018ea55c83cbbe"
format_stamp: "Formatted at 2026-08-11 09:15:00 on dist-test-slave-ncp9"
server_key: "ce9a2462da3f6cbbacef5422ce804be0"
server_key_iv: "dfaeffd40838fbd53f89e805b7dd269c"
server_key_version: "encryptionkey@0"
I20260811 09:15:00.053334 27672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:15:00.056721 27703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:00.058032 27672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:00.058182 27672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "dd7fd13bc5d647cfa1018ea55c83cbbe"
format_stamp: "Formatted at 2026-08-11 09:15:00 on dist-test-slave-ncp9"
server_key: "ce9a2462da3f6cbbacef5422ce804be0"
server_key_iv: "dfaeffd40838fbd53f89e805b7dd269c"
server_key_version: "encryptionkey@0"
I20260811 09:15:00.058364 27672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:00.075145 27672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:00.078823 27672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:00.079074 27672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:00.079962 27672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:00.081249 27672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:00.081329 27672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:00.081451 27672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:00.081506 27672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:00.111512 27672 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:39975
I20260811 09:15:00.111599 27917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:39975 every 8 connection(s)
I20260811 09:15:00.112792 27672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:15:00.116770 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 27672
I20260811 09:15:00.116997 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:15:00.117568 17473 external_mini_cluster.cc:1428] Setting key e4b00e48f015469186c57e08e4aa61ca
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:15:00.133533 27664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:00.115166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:59687 (local address 127.17.16.126:46867)
0811 09:15:00.115391 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.115397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.116514 (+ 1117us) server_negotiation.cc:407] Connection header received
0811 09:15:00.118862 (+ 2348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.118867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.118942 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.119047 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.120860 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.121629 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.123644 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.124076 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.124181 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:00.127988 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:00.127992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:00.127996 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:00.128029 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:00.130557 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.131251 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.131255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.131258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.131389 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.131811 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.131814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.131816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.132089 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:00.132231 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:00.132958 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 09:15:00.133203 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:15:00.134518 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:00.115604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46867 (local address 127.17.16.66:59687)
0811 09:15:00.116319 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:00.116378 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:00.118607 (+ 2229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:00.119230 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:00.119237 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:00.119899 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:00.120679 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:00.120704 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.121796 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.121802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:00.123496 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:00.123513 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.124299 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.124305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:00.124590 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:00.125299 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:00.125327 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:00.127764 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:00.130695 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:00.130704 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:00.130722 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:00.131136 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:00.131496 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:00.131501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:00.131505 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:00.131704 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:00.132223 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:00.132231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:00.132554 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 09:15:00.132817 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 09:15:00.133362 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":278,"threads_started":1}
I20260811 09:15:00.136062 27919 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46867
I20260811 09:15:00.136346 27919 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:00.137092 27919 heartbeater.cc:511] Master 127.17.16.126:46867 requested a full tablet report, sending...
I20260811 09:15:00.138612 27343 ts_manager.cc:194] Registered new tserver with Master: dd7fd13bc5d647cfa1018ea55c83cbbe (127.17.16.66:39975)
I20260811 09:15:00.139385 27343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:59687
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:15:00.172266 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46867
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:00.498622 27928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:00.498889 27928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:00.498999 27928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:00.506299 27928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:00.506400 27928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:00.506520 27928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:15:00.515921 27928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:00.517140 27928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:00.518401 27928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:00.530727 27935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:00.530646 27936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:00.530678 27938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:00.531529 27928 server_base.cc:1034] running on GCE node
I20260811 09:15:00.532083 27928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:00.533000 27928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:00.534207 27928 hybrid_clock.cc:648] HybridClock initialized: now 1786439700534162 us; error 39 us; skew 500 ppm
I20260811 09:15:00.534508 27928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:00.537451 27928 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:15:00.538740 27928 webserver.cc:460] Webserver started at http://127.17.16.67:36151/ using document root <none> and password file <none>
I20260811 09:15:00.539361 27928 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:00.539458 27928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:00.539708 27928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:00.542678 27928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "592a553c865b4cc8ad71e3055379c7eb"
format_stamp: "Formatted at 2026-08-11 09:15:00 on dist-test-slave-ncp9"
server_key: "46ff59ac4494f28a61c98b95c36a131d"
server_key_iv: "63055dd3b1270d29b4fc8cc470f8c578"
server_key_version: "encryptionkey@0"
I20260811 09:15:00.543394 27928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "592a553c865b4cc8ad71e3055379c7eb"
format_stamp: "Formatted at 2026-08-11 09:15:00 on dist-test-slave-ncp9"
server_key: "46ff59ac4494f28a61c98b95c36a131d"
server_key_iv: "63055dd3b1270d29b4fc8cc470f8c578"
server_key_version: "encryptionkey@0"
I20260811 09:15:00.548270 27928 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:15:00.551610 27955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:00.552866 27928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:15:00.553043 27928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "592a553c865b4cc8ad71e3055379c7eb"
format_stamp: "Formatted at 2026-08-11 09:15:00 on dist-test-slave-ncp9"
server_key: "46ff59ac4494f28a61c98b95c36a131d"
server_key_iv: "63055dd3b1270d29b4fc8cc470f8c578"
server_key_version: "encryptionkey@0"
I20260811 09:15:00.553231 27928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:00.565559 27928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:00.569205 27928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:00.569470 27928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:00.570384 27928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:00.571776 27928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:00.571858 27928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:00.571972 27928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:00.572021 27928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:00.600625 27928 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:41955
I20260811 09:15:00.600667 28132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:41955 every 8 connection(s)
I20260811 09:15:00.601850 27928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:15:00.605994 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 27928
I20260811 09:15:00.606215 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:15:00.606731 17473 external_mini_cluster.cc:1428] Setting key 6cd573866ebed8a04be3a1bfe9403937
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:15:00.621354 27664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:00.604266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:59037 (local address 127.17.16.126:46867)
0811 09:15:00.604441 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.604446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.605605 (+ 1159us) server_negotiation.cc:407] Connection header received
0811 09:15:00.607895 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.607899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.608015 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.608116 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.609950 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.610713 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.612222 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.612656 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.612764 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:00.616065 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:00.616070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:00.616073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:00.616107 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:00.618410 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.619099 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.619103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.619105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.619242 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.619662 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.619665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.619668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.619938 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:00.620069 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:00.620774 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 09:15:00.621023 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:15:00.622349 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:00.604667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46867 (local address 127.17.16.67:59037)
0811 09:15:00.605435 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:00.605473 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:00.607627 (+ 2154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:00.608337 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:00.608344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:00.609006 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:00.609785 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:00.609806 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.610864 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.610868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:00.612076 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:00.612089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.612874 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.612881 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:00.613127 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:00.613704 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:00.613724 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:00.615869 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:00.618539 (+ 2670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:00.618548 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:00.618565 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:00.618977 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:00.619347 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:00.619352 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:00.619355 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:00.619538 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:00.620060 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:00.620068 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:00.620375 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 09:15:00.620616 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 09:15:00.621159 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":303,"threads_started":1}
I20260811 09:15:00.623492 28134 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46867
I20260811 09:15:00.623795 28134 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:00.624460 28134 heartbeater.cc:511] Master 127.17.16.126:46867 requested a full tablet report, sending...
I20260811 09:15:00.625818 27343 ts_manager.cc:194] Registered new tserver with Master: 592a553c865b4cc8ad71e3055379c7eb (127.17.16.67:41955)
I20260811 09:15:00.626577 27343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:59037
I20260811 09:15:00.629294 27658 heartbeater.cc:503] Master 127.17.16.126:46867 was elected leader, sending a full tablet report...
I20260811 09:15:00.634109 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/test-admin.keytab.
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:00.666018 17473 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 09:15:00.670982 28145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:00.668710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39256 (local address 127.17.16.65:39023)
0811 09:15:00.669325 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.669332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.669359 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:15:00.669422 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.669429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.669649 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.669781 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.670564 (+ 783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39256: BlockingRecv error: recv got EOF from 127.0.0.1:39256 (error 108)
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":322,"threads_started":1}
W20260811 09:15:00.671084 17473 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5cc' of type FILE
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:00.672606 17473 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:00.673974 17473 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:15:00.688612 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:00.675546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39266 (local address 127.17.16.65:39023)
0811 09:15:00.675749 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.675754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.675873 (+ 119us) server_negotiation.cc:407] Connection header received
0811 09:15:00.676018 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.676021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.676076 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.676158 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.677065 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.678059 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.679187 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.679730 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.679823 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:00.683218 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:00.683230 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:00.683246 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:00.683278 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:00.685874 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.686553 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.686561 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.686563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.686684 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.687042 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.687046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.687048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.687430 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:00.687707 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:00.688072 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 09:15:00.688348 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 09:15:00.694360 28145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:00.692546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.17.16.65:39023)
0811 09:15:00.692690 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.692696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.693050 (+ 354us) server_negotiation.cc:407] Connection header received
0811 09:15:00.693197 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.693201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.693268 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.693343 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.694155 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39274: BlockingRecv error: recv got EOF from 127.0.0.1:39274 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 09:15:00.694584 17473 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5cc' of type FILE
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:00.696167 17473 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:00.697474 17473 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:15:00.711043 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:00.698902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39282 (local address 127.17.16.65:39023)
0811 09:15:00.699084 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.699090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.699250 (+ 160us) server_negotiation.cc:407] Connection header received
0811 09:15:00.699374 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.699378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.699435 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.699509 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.700455 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.701231 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.702436 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.702846 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.702932 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:00.706451 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:00.706456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:00.706459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:00.706490 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:00.708866 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.709472 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.709475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.709478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.709593 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.709886 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.709889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.709891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.710131 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:00.710326 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:00.710600 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 09:15:00.710756 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 09:15:00.715013 28145 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:00.713577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.17.16.65:39023)
0811 09:15:00.713717 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.713721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.713980 (+ 259us) server_negotiation.cc:407] Connection header received
0811 09:15:00.714099 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.714102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.714168 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.714241 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.714804 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39298: BlockingRecv error: recv got EOF from 127.0.0.1:39298 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 09:15:00.715276 17473 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439649204149-17473-0/krb5kdc/krb5cc' of type FILE
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:00.716634 17473 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:00.717852 17473 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:15:00.730648 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:00.719056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39304 (local address 127.17.16.65:39023)
0811 09:15:00.719262 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.719268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.719431 (+ 163us) server_negotiation.cc:407] Connection header received
0811 09:15:00.719536 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.719539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.719592 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.719661 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.720497 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.721250 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.722431 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.722823 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.722909 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:00.726285 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:00.726290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:00.726293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:00.726323 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:00.728542 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.729173 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.729178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.729180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.729324 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.729600 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.729603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.729605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.729864 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:00.730082 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:00.730239 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 09:15:00.730393 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 09:15:00.732009 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 27451
I20260811 09:15:00.742774 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:00.732626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.17.16.65:39023)
0811 09:15:00.732761 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.732765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.733151 (+ 386us) server_negotiation.cc:407] Connection header received
0811 09:15:00.733255 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.733258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.733308 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.733376 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.734166 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.734798 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.735894 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.736305 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.736395 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:00.738293 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:00.738298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:00.738301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:00.738328 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:00.740599 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.741200 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.741203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.741205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.741322 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.741631 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.741635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.741638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.741953 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:00.742147 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:00.742375 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 09:15:00.742523 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:15:00.743330 27652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:00.826452 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 27451
I20260811 09:15:00.850821 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 27672
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:15:00.865446 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:00.851712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55232 (local address 127.17.16.66:39975)
0811 09:15:00.852198 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.852205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.852230 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:15:00.852317 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.852324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.852542 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.852713 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.853871 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.854915 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.856087 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.856665 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.856789 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:00.860435 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:00.860446 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:00.860459 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:00.860508 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:00.863100 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.863700 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.863712 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.863714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.863835 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.864125 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.864128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.864130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.864444 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:00.864674 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:00.864965 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 09:15:00.865156 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":235,"threads_started":1}
I20260811 09:15:00.866658 27907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:00.947561 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 27672
I20260811 09:15:00.971036 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 27928
Aug 11 09:15:00 dist-test-slave-ncp9 krb5kdc[27265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:15:00.986137 28168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:00.972070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.17.16.67:41955)
0811 09:15:00.972779 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:00.972786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:00.972815 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:15:00.972877 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:00.972883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:00.973169 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:00.973298 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:00.974275 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.975281 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.976440 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:00.977037 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:00.977133 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:00.980769 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:00.980781 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:00.980795 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:00.980827 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:00.983528 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.984153 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.984159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.984161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.984278 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:00.984586 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:00.984590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:00.984591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:00.985094 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:00.985324 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:00.985590 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 09:15:00.985823 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":390,"threads_started":1}
I20260811 09:15:00.987401 28128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:01.069012 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 27928
I20260811 09:15:01.089422 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 27295
I20260811 09:15:01.090657 27402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:01.191994 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 27295
2026-08-11T09:15:01Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4056 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:15:01 dist-test-slave-ncp9 krb5kdc[28177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:15:01 dist-test-slave-ncp9 krb5kdc[28177](info): set up 2 sockets
Aug 11 09:15:01 dist-test-slave-ncp9 krb5kdc[28177](info): commencing operation
krb5kdc: starting...
W20260811 09:15:02.408842 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:15:02 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:15:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:15:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.126.keytab.
Entry for principal oryx/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.126.keytab.
I20260811 09:15:02.644680 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:39045
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.126.keytab
--principal=oryx/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:39045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:02.965441 28200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:02.965746 28200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:02.965885 28200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:02.973043 28200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:15:02.973160 28200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:02.973239 28200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:02.973315 28200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:15:02.973373 28200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:15:02.983526 28200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:39045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.126.keytab
--principal=oryx/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:39045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:02.984862 28200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:02.986270 28200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:02.997824 28208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:02.997936 28206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:02.997872 28211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:02.999131 28200 server_base.cc:1034] running on GCE node
I20260811 09:15:02.999866 28200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:03.001262 28200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:03.002460 28200 hybrid_clock.cc:648] HybridClock initialized: now 1786439703002415 us; error 33 us; skew 500 ppm
I20260811 09:15:03.002797 28200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:03 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439703, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:03.005733 28200 init.cc:377] Logged in from keytab as oryx/127.17.16.126@KRBTEST.COM (short username oryx)
I20260811 09:15:03.007184 28200 webserver.cc:460] Webserver started at http://127.17.16.126:38923/ using document root <none> and password file <none>
I20260811 09:15:03.007851 28200 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:03.007968 28200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:03.008255 28200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:03.011327 28200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "31eca697b43b43199c0142a966b3b61c"
format_stamp: "Formatted at 2026-08-11 09:15:03 on dist-test-slave-ncp9"
server_key: "717ccc335cc6e8a579e343dbd0960120"
server_key_iv: "fd46fe5fbcd03c975c5a6fbb8409f9cb"
server_key_version: "encryptionkey@0"
I20260811 09:15:03.012065 28200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "31eca697b43b43199c0142a966b3b61c"
format_stamp: "Formatted at 2026-08-11 09:15:03 on dist-test-slave-ncp9"
server_key: "717ccc335cc6e8a579e343dbd0960120"
server_key_iv: "fd46fe5fbcd03c975c5a6fbb8409f9cb"
server_key_version: "encryptionkey@0"
I20260811 09:15:03.017354 28200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260811 09:15:03.020714 28225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:03.022015 28200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:03.022182 28200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "31eca697b43b43199c0142a966b3b61c"
format_stamp: "Formatted at 2026-08-11 09:15:03 on dist-test-slave-ncp9"
server_key: "717ccc335cc6e8a579e343dbd0960120"
server_key_iv: "fd46fe5fbcd03c975c5a6fbb8409f9cb"
server_key_version: "encryptionkey@0"
I20260811 09:15:03.022373 28200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:03.033998 28200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:03.037484 28200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:03.037735 28200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:03.054788 28200 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:39045
I20260811 09:15:03.054821 28307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:39045 every 8 connection(s)
I20260811 09:15:03.056208 28200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:15:03.059916 28310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:03.065219 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 28200
I20260811 09:15:03.065474 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:15:03.066268 17473 external_mini_cluster.cc:1428] Setting key 5b56e61976ecc28f53c969f1fabc2b0a
I20260811 09:15:03.072674 28310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c: Bootstrap starting.
I20260811 09:15:03.076483 28310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:03.077699 28310 log.cc:824] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:03.081065 28310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c: No bootstrap required, opened a new log
Aug 11 09:15:03 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:03.088619 28310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } }
I20260811 09:15:03.089001 28310 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:03.089119 28310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31eca697b43b43199c0142a966b3b61c, State: Initialized, Role: FOLLOWER
I20260811 09:15:03.089766 28315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:03.068412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49998 (local address 127.17.16.126:39045)
0811 09:15:03.069262 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:03.069275 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:03.069326 (+ 51us) server_negotiation.cc:407] Connection header received
0811 09:15:03.071504 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:03.071536 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:03.072053 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:03.072509 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:03.074318 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:03.075617 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:03.077310 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:03.078005 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:03.078152 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:03.082654 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:03.082662 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:03.082678 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:03.082722 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:03.085765 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:03.086422 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:03.086428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:03.086435 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:03.086612 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:03.086993 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:03.086997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:03.086999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:03.087536 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:03.087841 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:03.088221 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 09:15:03.088627 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":386,"threads_started":1}
I20260811 09:15:03.089727 28310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } }
I20260811 09:15:03.089977 28310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:15:03.090103 28310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:15:03.090284 28310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:03.092625 28310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } }
I20260811 09:15:03.093152 28310 leader_election.cc:302] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31eca697b43b43199c0142a966b3b61c; no voters:
I20260811 09:15:03.093708 28310 leader_election.cc:288] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:15:03.093817 28317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:03.094473 28317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 1 LEADER]: Becoming Leader. State: Replica: 31eca697b43b43199c0142a966b3b61c, State: Running, Role: LEADER
I20260811 09:15:03.094973 28317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } }
I20260811 09:15:03.095685 28310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:15:03.097348 28319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31eca697b43b43199c0142a966b3b61c. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } }
I20260811 09:15:03.097731 28319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:03.098595 28327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:15:03.100376 28318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } }
I20260811 09:15:03.100791 28318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:03.108423 28327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:15:03.117126 28327 catalog_manager.cc:1347] Generated new cluster ID: 363259a4cc274759bc3e778fde34f144
I20260811 09:15:03.117273 28327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:15:03.135186 28327 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:15:03.136485 28327 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:15:03.143661 28327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c: Generated new TSK 0
I20260811 09:15:03.144404 28327 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab.
Entry for principal oryx/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab.
I20260811 09:15:03.210459 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:39045
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab
--principal=oryx/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:03.538192 28358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:03.538447 28358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:03.538582 28358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:03.545691 28358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:03.545804 28358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:03.545960 28358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:15:03.555248 28358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab
--principal=oryx/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:39045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:03.556411 28358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:03.557675 28358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:03.569873 28372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:03.569939 28367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:03.569911 28369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:03.570745 28358 server_base.cc:1034] running on GCE node
I20260811 09:15:03.571318 28358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:03.572201 28358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:03.573395 28358 hybrid_clock.cc:648] HybridClock initialized: now 1786439703573337 us; error 49 us; skew 500 ppm
I20260811 09:15:03.573684 28358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:03 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439703, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:03.576609 28358 init.cc:377] Logged in from keytab as oryx/127.17.16.65@KRBTEST.COM (short username oryx)
I20260811 09:15:03.577935 28358 webserver.cc:460] Webserver started at http://127.17.16.65:36051/ using document root <none> and password file <none>
I20260811 09:15:03.578553 28358 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:03.578657 28358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:03.578910 28358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:03.581890 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "be3e7b06847640edaf6a5a221d39a9f2"
format_stamp: "Formatted at 2026-08-11 09:15:03 on dist-test-slave-ncp9"
server_key: "7f2682dadbae68d0cd074b5edc7d39c7"
server_key_iv: "53d5def81eed6e0a5cca7a106f4fd0b4"
server_key_version: "encryptionkey@0"
I20260811 09:15:03.582615 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "be3e7b06847640edaf6a5a221d39a9f2"
format_stamp: "Formatted at 2026-08-11 09:15:03 on dist-test-slave-ncp9"
server_key: "7f2682dadbae68d0cd074b5edc7d39c7"
server_key_iv: "53d5def81eed6e0a5cca7a106f4fd0b4"
server_key_version: "encryptionkey@0"
I20260811 09:15:03.587414 28358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:15:03.590632 28380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:03.591912 28358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 09:15:03.592053 28358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "be3e7b06847640edaf6a5a221d39a9f2"
format_stamp: "Formatted at 2026-08-11 09:15:03 on dist-test-slave-ncp9"
server_key: "7f2682dadbae68d0cd074b5edc7d39c7"
server_key_iv: "53d5def81eed6e0a5cca7a106f4fd0b4"
server_key_version: "encryptionkey@0"
I20260811 09:15:03.592238 28358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:03.609866 28358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:03.613770 28358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:03.614012 28358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:03.614892 28358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:03.616214 28358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:03.616286 28358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:03.616382 28358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:03.616425 28358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:03.646373 28358 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:45197
I20260811 09:15:03.646418 28575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:45197 every 8 connection(s)
I20260811 09:15:03.647647 28358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:15:03.653429 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 28358
I20260811 09:15:03.653693 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:15:03.654542 17473 external_mini_cluster.cc:1428] Setting key 550ca8f0f18442fae72d6174f65713ed
Aug 11 09:15:03 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439703, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:03.668787 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:03.650258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:50971 (local address 127.17.16.126:39045)
0811 09:15:03.650675 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:03.650680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:03.651693 (+ 1013us) server_negotiation.cc:407] Connection header received
0811 09:15:03.653674 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:03.653679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:03.653750 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:03.653861 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:03.655809 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:03.656829 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:03.659011 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:03.659398 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:03.659487 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:03.663420 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:03.663425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:03.663428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:03.663461 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:03.665854 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:03.666601 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:03.666604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:03.666607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:03.666733 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:03.667131 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:03.667135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:03.667138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:03.667474 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:03.667618 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:03.668231 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 09:15:03.668431 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":198,"threads_started":1}
I20260811 09:15:03.669559 28588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:03.650849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.65:50971)
0811 09:15:03.651538 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:03.651576 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:03.653382 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:03.654160 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:03.654171 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:03.654894 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:03.655617 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:03.655642 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:03.657029 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:03.657035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:03.658877 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:03.658893 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:03.659615 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:03.659623 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:03.659893 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:03.660553 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:03.660585 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:03.663205 (+ 2620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:03.665979 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:03.665986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:03.666002 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:03.666511 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:03.666840 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:03.666844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:03.666846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:03.667034 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:03.667576 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:03.667581 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:03.667875 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 09:15:03.668106 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 09:15:03.668474 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":261,"threads_started":1}
I20260811 09:15:03.670921 28576 heartbeater.cc:347] Connected to a master server at 127.17.16.126:39045
I20260811 09:15:03.671229 28576 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:03.672147 28576 heartbeater.cc:511] Master 127.17.16.126:39045 requested a full tablet report, sending...
I20260811 09:15:03.674249 28252 ts_manager.cc:194] Registered new tserver with Master: be3e7b06847640edaf6a5a221d39a9f2 (127.17.16.65:45197)
I20260811 09:15:03.675649 28252 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:50971
WARNING: no policy specified for oryx/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab.
Entry for principal oryx/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab.
I20260811 09:15:03.710259 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:39045
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab
--principal=oryx/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:04.038772 28591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:04.039049 28591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:04.039186 28591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:04.046208 28591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:04.046319 28591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:04.046476 28591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:15:04.057767 28591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab
--principal=oryx/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:39045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:04.059283 28591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:04.061058 28591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:04.075121 28600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:04.075243 28598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:04.077914 28603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:04.078127 28591 server_base.cc:1034] running on GCE node
I20260811 09:15:04.078727 28591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:04.079617 28591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:04.080822 28591 hybrid_clock.cc:648] HybridClock initialized: now 1786439704080768 us; error 44 us; skew 500 ppm
I20260811 09:15:04.081140 28591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:04 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439704, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:04.087092 28591 init.cc:377] Logged in from keytab as oryx/127.17.16.66@KRBTEST.COM (short username oryx)
I20260811 09:15:04.088622 28591 webserver.cc:460] Webserver started at http://127.17.16.66:35379/ using document root <none> and password file <none>
I20260811 09:15:04.089380 28591 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:04.089485 28591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:04.089756 28591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:04.092804 28591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "dde75c9995e94b5cbbe56c74a12a9445"
format_stamp: "Formatted at 2026-08-11 09:15:04 on dist-test-slave-ncp9"
server_key: "25d88c56ed0dbd18f4ed11b28475c660"
server_key_iv: "61c451ae1bebe10e3b22d1739b1fc5f1"
server_key_version: "encryptionkey@0"
I20260811 09:15:04.093714 28591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "dde75c9995e94b5cbbe56c74a12a9445"
format_stamp: "Formatted at 2026-08-11 09:15:04 on dist-test-slave-ncp9"
server_key: "25d88c56ed0dbd18f4ed11b28475c660"
server_key_iv: "61c451ae1bebe10e3b22d1739b1fc5f1"
server_key_version: "encryptionkey@0"
I20260811 09:15:04.098903 28591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:15:04.102205 28612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:04.103488 28591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:15:04.103618 28591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "dde75c9995e94b5cbbe56c74a12a9445"
format_stamp: "Formatted at 2026-08-11 09:15:04 on dist-test-slave-ncp9"
server_key: "25d88c56ed0dbd18f4ed11b28475c660"
server_key_iv: "61c451ae1bebe10e3b22d1739b1fc5f1"
server_key_version: "encryptionkey@0"
I20260811 09:15:04.103792 28591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:04.122275 28591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:04.125939 28591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:04.126240 28591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:04.127353 28591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:04.128644 28591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:04.128717 28591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:04.128808 28591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:04.128854 28591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:04.158517 28591 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:39355
I20260811 09:15:04.158572 28792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:39355 every 8 connection(s)
I20260811 09:15:04.159718 28591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:15:04.164660 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 28591
I20260811 09:15:04.164842 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:15:04.165328 17473 external_mini_cluster.cc:1428] Setting key 0ff2a67cc7279732dec73b98ae5fec4a
Aug 11 09:15:04 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439704, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:04.180106 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:04.162101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:49505 (local address 127.17.16.126:39045)
0811 09:15:04.162334 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:04.162340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:04.163453 (+ 1113us) server_negotiation.cc:407] Connection header received
0811 09:15:04.165286 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:04.165292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:04.165384 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:04.165492 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:04.167191 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:04.167931 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:04.170151 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:04.170574 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:04.170670 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:04.174501 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:04.174506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:04.174510 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:04.174544 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:04.177102 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:04.177827 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:04.177832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:04.177834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:04.177966 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:04.178399 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:04.178403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:04.178405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:04.178699 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:04.178832 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:04.179527 (+ 695us) server_negotiation.cc:299] Negotiation successful
0811 09:15:04.179762 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:15:04.181155 28797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:04.162626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.66:49505)
0811 09:15:04.163297 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:04.163342 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:04.165029 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:04.165798 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:04.165806 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:04.166411 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:04.166995 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:04.167023 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:04.168105 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:04.168110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:04.169997 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:04.170014 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:04.170792 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:04.170799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:04.171079 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:04.171760 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:04.171794 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:04.174266 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:04.177244 (+ 2978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:04.177254 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:04.177272 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:04.177699 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:04.178077 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:04.178082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:04.178086 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:04.178282 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:04.178825 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:04.178832 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:04.179164 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 09:15:04.179407 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 09:15:04.179959 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":248,"threads_started":1}
I20260811 09:15:04.182787 28794 heartbeater.cc:347] Connected to a master server at 127.17.16.126:39045
I20260811 09:15:04.183075 28794 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:04.183830 28794 heartbeater.cc:511] Master 127.17.16.126:39045 requested a full tablet report, sending...
I20260811 09:15:04.185424 28252 ts_manager.cc:194] Registered new tserver with Master: dde75c9995e94b5cbbe56c74a12a9445 (127.17.16.66:39355)
I20260811 09:15:04.186515 28252 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:49505
WARNING: no policy specified for oryx/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab.
Entry for principal oryx/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab.
I20260811 09:15:04.217839 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:39045
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab
--principal=oryx/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:04.539716 28802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:04.539991 28802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:04.540093 28802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:04.547237 28802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:04.547338 28802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:04.547457 28802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:15:04.560178 28802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab
--principal=oryx/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:39045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:04.561559 28802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:04.562876 28802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:04.575595 28810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:04.575520 28811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:04.575569 28813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:04.576807 28802 server_base.cc:1034] running on GCE node
I20260811 09:15:04.577489 28802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:04.578436 28802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:04.579636 28802 hybrid_clock.cc:648] HybridClock initialized: now 1786439704579582 us; error 45 us; skew 500 ppm
I20260811 09:15:04.579937 28802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:04 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439704, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:04.583086 28802 init.cc:377] Logged in from keytab as oryx/127.17.16.67@KRBTEST.COM (short username oryx)
I20260811 09:15:04.584429 28802 webserver.cc:460] Webserver started at http://127.17.16.67:44631/ using document root <none> and password file <none>
I20260811 09:15:04.585173 28802 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:04.585294 28802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:04.585587 28802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:04.588404 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "9170330144184e8385e0338b4b2521c2"
format_stamp: "Formatted at 2026-08-11 09:15:04 on dist-test-slave-ncp9"
server_key: "6ec055dd7e406c9b368b569c123315c0"
server_key_iv: "e7da8a08ae624f8b36851452d02993d2"
server_key_version: "encryptionkey@0"
I20260811 09:15:04.589202 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "9170330144184e8385e0338b4b2521c2"
format_stamp: "Formatted at 2026-08-11 09:15:04 on dist-test-slave-ncp9"
server_key: "6ec055dd7e406c9b368b569c123315c0"
server_key_iv: "e7da8a08ae624f8b36851452d02993d2"
server_key_version: "encryptionkey@0"
I20260811 09:15:04.593956 28802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:15:04.597368 28823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:04.598623 28802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:15:04.598773 28802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "9170330144184e8385e0338b4b2521c2"
format_stamp: "Formatted at 2026-08-11 09:15:04 on dist-test-slave-ncp9"
server_key: "6ec055dd7e406c9b368b569c123315c0"
server_key_iv: "e7da8a08ae624f8b36851452d02993d2"
server_key_version: "encryptionkey@0"
I20260811 09:15:04.598945 28802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:04.614311 28802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:04.618016 28802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:04.618283 28802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:04.619184 28802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:04.620488 28802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:04.620568 28802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:04.620682 28802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:04.620736 28802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:04.652374 28802 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:40501
I20260811 09:15:04.652417 29012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:40501 every 8 connection(s)
I20260811 09:15:04.653616 28802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:15:04.660243 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 28802
I20260811 09:15:04.660418 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:15:04.660833 17473 external_mini_cluster.cc:1428] Setting key 44ea7ff7546a46b11ca17cb638193fea
Aug 11 09:15:04 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439704, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:04.671844 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:04.656042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:39235 (local address 127.17.16.126:39045)
0811 09:15:04.656213 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:04.656217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:04.657410 (+ 1193us) server_negotiation.cc:407] Connection header received
0811 09:15:04.659171 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:04.659178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:04.659252 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:04.659343 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:04.660826 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:04.661657 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:04.663154 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:04.663580 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:04.663688 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:04.666813 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:04.666818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:04.666821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:04.666852 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:04.669068 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:04.669707 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:04.669712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:04.669716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:04.669843 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:04.670246 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:04.670252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:04.670255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:04.670513 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:04.670657 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:04.671296 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 09:15:04.671505 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:15:04.672683 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:04.656517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.67:39235)
0811 09:15:04.657251 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:04.657287 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:04.658867 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:04.659586 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:04.659595 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:04.660072 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:04.660660 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:04.660679 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:04.661809 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:04.661813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:04.663003 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:04.663016 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:04.663791 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:04.663798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:04.664029 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:04.664569 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:04.664590 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:04.666619 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:04.669194 (+ 2575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:04.669203 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:04.669217 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:04.669599 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:04.669945 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:04.669950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:04.669954 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:04.670142 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:04.670647 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:04.670654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:04.670940 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 09:15:04.671185 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 09:15:04.671662 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":292,"threads_started":1}
I20260811 09:15:04.673931 29013 heartbeater.cc:347] Connected to a master server at 127.17.16.126:39045
I20260811 09:15:04.674245 29013 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:04.675138 29013 heartbeater.cc:511] Master 127.17.16.126:39045 requested a full tablet report, sending...
I20260811 09:15:04.676515 28252 ts_manager.cc:194] Registered new tserver with Master: 9170330144184e8385e0338b4b2521c2 (127.17.16.67:40501)
I20260811 09:15:04.677317 28252 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:39235
I20260811 09:15:04.679000 28576 heartbeater.cc:503] Master 127.17.16.126:39045 was elected leader, sending a full tablet report...
I20260811 09:15:04.687578 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.16.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.125.keytab.
Entry for principal oryx/127.17.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.125.keytab.
I20260811 09:15:04.742694 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.125:38133
--webserver_interface=127.17.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.125.keytab
--principal=oryx/127.17.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:39045,127.17.16.125:38133
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:05.070209 29022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:05.070524 29022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:05.070681 29022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:05.078028 29022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:15:05.078148 29022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:05.078236 29022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:05.078327 29022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:15:05.078398 29022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:15:05.088744 29022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:39045,127.17.16.125:38133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.125.keytab
--principal=oryx/127.17.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.125:38133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
W20260811 09:15:05.089140 29022 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.16.126:39045,127.17.16.125:38133'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 09:15:05.090222 29022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:05.091554 29022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:05.103425 29033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:05.103516 29031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:05.103515 29029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:05.104984 29022 server_base.cc:1034] running on GCE node
I20260811 09:15:05.105827 29022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:05.107215 29022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:05.108494 29022 hybrid_clock.cc:648] HybridClock initialized: now 1786439705108410 us; error 43 us; skew 500 ppm
I20260811 09:15:05.108829 29022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439705, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:05.111951 29022 init.cc:377] Logged in from keytab as oryx/127.17.16.125@KRBTEST.COM (short username oryx)
I20260811 09:15:05.113389 29022 webserver.cc:460] Webserver started at http://127.17.16.125:46283/ using document root <none> and password file <none>
I20260811 09:15:05.114070 29022 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:05.114181 29022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:05.114617 29022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:05.117740 29022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data/instance:
uuid: "8f9ed5de1a944954a8c1e5361d98719e"
format_stamp: "Formatted at 2026-08-11 09:15:05 on dist-test-slave-ncp9"
server_key: "41a4938bce315085cf6072a515c4a870"
server_key_iv: "a0fb898be1e6b252f859ee206c345099"
server_key_version: "encryptionkey@0"
I20260811 09:15:05.118538 29022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal/instance:
uuid: "8f9ed5de1a944954a8c1e5361d98719e"
format_stamp: "Formatted at 2026-08-11 09:15:05 on dist-test-slave-ncp9"
server_key: "41a4938bce315085cf6072a515c4a870"
server_key_iv: "a0fb898be1e6b252f859ee206c345099"
server_key_version: "encryptionkey@0"
I20260811 09:15:05.123768 29022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:15:05.127367 29046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:05.128651 29022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:15:05.128803 29022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal
uuid: "8f9ed5de1a944954a8c1e5361d98719e"
format_stamp: "Formatted at 2026-08-11 09:15:05 on dist-test-slave-ncp9"
server_key: "41a4938bce315085cf6072a515c4a870"
server_key_iv: "a0fb898be1e6b252f859ee206c345099"
server_key_version: "encryptionkey@0"
I20260811 09:15:05.129077 29022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:05.141806 29022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:05.145418 29022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:05.145635 29022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:05.162667 29022 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.125:38133
I20260811 09:15:05.162626 29128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.125:38133 every 8 connection(s)
I20260811 09:15:05.164301 29022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
I20260811 09:15:05.167908 29130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:05.174623 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 29022
I20260811 09:15:05.174693 29130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439705, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.125@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.125@KRBTEST.COM
I20260811 09:15:05.189147 28794 heartbeater.cc:503] Master 127.17.16.126:39045 was elected leader, sending a full tablet report...
I20260811 09:15:05.196626 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.178041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50002 (local address 127.17.16.126:39045)
0811 09:15:05.179197 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.179202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.179219 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:15:05.179963 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.179968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.180038 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.180130 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.181420 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.182176 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.184044 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.184691 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.184811 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.188492 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.188499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.188504 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.188542 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.192159 (+ 3617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.193025 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.193032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.193036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.193193 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.193652 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.193655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.193658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.193999 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.194147 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.194904 (+ 757us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.196236 (+ 1332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1057,"thread_start_us":947,"threads_started":1}
I20260811 09:15:05.196676 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.176997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58598 (local address 127.17.16.125:38133)
0811 09:15:05.177619 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.177628 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.177665 (+ 37us) server_negotiation.cc:407] Connection header received
0811 09:15:05.179712 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.179733 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.179992 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.180169 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.181709 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.183351 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.184809 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.185585 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.185676 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.190043 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.190052 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.190063 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.190094 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.192834 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.193488 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.193494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.193496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.193624 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.193982 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.193986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.193987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.194477 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.194783 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.195072 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.195379 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"spinlock_wait_cycles":9472,"thread_start_us":270,"threads_started":1}
I20260811 09:15:05.196703 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.177924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:50002)
0811 09:15:05.178603 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.178644 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.179725 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.180214 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.180221 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.180837 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.181258 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.181275 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.182373 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.182379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.183906 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.183918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.184940 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.184948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.185220 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.185920 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.185940 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.188299 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.192295 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.192307 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.192319 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.192873 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.193308 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.193314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.193318 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.193521 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.194135 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.194143 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.194500 (+ 357us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.194767 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.195863 (+ 1096us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":161,"spinlock_wait_cycles":12672,"thread_start_us":291,"threads_started":1}
I20260811 09:15:05.198915 29130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439705, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.125@KRBTEST.COM for oryx/127.17.16.125@KRBTEST.COM
I20260811 09:15:05.210650 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.199469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:38133 (local address 127.0.0.1:58604)
0811 09:15:05.199609 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.199626 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.199706 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.200286 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.200289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.200564 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.200695 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.200702 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.201651 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.201654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.202528 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.202539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.203219 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.203222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.203396 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.203825 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.203840 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.205788 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.208556 (+ 2768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.208559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.208561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.208951 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.209273 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.209276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.209278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.209422 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.209967 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.209970 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.210099 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.210205 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.210411 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 09:15:05.210805 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.199889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58604 (local address 127.17.16.125:38133)
0811 09:15:05.200060 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.200063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.200078 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:15:05.200116 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.200119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.200169 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.200246 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.200808 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.201520 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.202657 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.203094 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.203204 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.205978 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.205983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.205986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.206015 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.208455 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.209054 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.209058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.209060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.209181 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.209577 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.209581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.209583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.209858 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.209998 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.210320 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.210530 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":58}
I20260811 09:15:05.215230 29130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Bootstrap starting.
I20260811 09:15:05.218245 29130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:05.219211 29130 log.cc:824] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:05.221779 29130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: No bootstrap required, opened a new log
I20260811 09:15:05.228557 29130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } }
I20260811 09:15:05.228870 29130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:05.228981 29130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9ed5de1a944954a8c1e5361d98719e, State: Initialized, Role: FOLLOWER
I20260811 09:15:05.229530 29130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } }
I20260811 09:15:05.230371 29142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } } }
I20260811 09:15:05.230720 29142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:15:05.231236 29130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:15:05.239185 29156 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:15:05.239306 29156 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:15:05.242967 29022 master_runner.cc:428] Detected that this master 8f9ed5de1a944954a8c1e5361d98719e is joining an existing cluster
I20260811 09:15:05.243048 29022 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.16.125:38133
I20260811 09:15:05.262954 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.246646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.17.16.126:39045)
0811 09:15:05.246856 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.246863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.247345 (+ 482us) server_negotiation.cc:407] Connection header received
0811 09:15:05.247750 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.247757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.247861 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.247957 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.249321 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.250160 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.252160 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.252847 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.252967 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.255578 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.255585 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.255589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.255631 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.259055 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.260520 (+ 1465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.260527 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.260530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.260708 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.261197 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.261202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.261206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.261597 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.261719 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.262365 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.262618 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:15:05.262995 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.247495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58618 (local address 127.17.16.125:38133)
0811 09:15:05.247650 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.247656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.248289 (+ 633us) server_negotiation.cc:407] Connection header received
0811 09:15:05.248460 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.248465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.248537 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.248638 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.249892 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.251001 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.252985 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.253692 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.254064 (+ 372us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.256617 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.256623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.256627 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.256660 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.260061 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.260849 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.260854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.260857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.261064 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.261466 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.261470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.261473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.261820 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.261927 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.262402 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.262605 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":46}
I20260811 09:15:05.263912 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.246591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:50006)
0811 09:15:05.247246 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.247266 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.247464 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.248071 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.248076 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.248789 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.249178 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.249189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.250333 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.250342 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.251988 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.252003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.253317 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.253323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.253537 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.254217 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.254243 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.255337 (+ 1094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.259208 (+ 3871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.259213 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.259218 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.260357 (+ 1139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.260855 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.260860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.260863 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.261074 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.261817 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.261821 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.262019 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.263158 (+ 1139us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.263483 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":59,"spinlock_wait_cycles":6272,"thread_start_us":326,"threads_started":1}
I20260811 09:15:05.264535 29174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.247576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:38133 (local address 127.0.0.1:58618)
0811 09:15:05.248180 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.248201 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.248327 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.248679 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.248685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.249408 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.249737 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.249750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.251226 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.251231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.252791 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.252806 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.254391 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.254397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.254561 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.255164 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.255188 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.256451 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.260177 (+ 3726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.260181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.260185 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.260735 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.261197 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.261200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.261204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.261353 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.262019 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.262023 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.262165 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.263200 (+ 1035us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.263807 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":334,"thread_start_us":268,"threads_started":1}
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439703, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.126@KRBTEST.COM for oryx/127.17.16.125@KRBTEST.COM
WARNING: no policy specified for oryx/127.17.16.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.124@KRBTEST.COM" created.
I20260811 09:15:05.286769 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.270340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58624 (local address 127.17.16.125:38133)
0811 09:15:05.270521 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.270527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.273569 (+ 3042us) server_negotiation.cc:407] Connection header received
0811 09:15:05.273909 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.273913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.274004 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.274092 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.275196 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.275891 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.277203 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.277749 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.277859 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.280886 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.280914 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.280918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.280960 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.283653 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.284370 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.284376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.284380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.284548 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.284993 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.285002 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.285006 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.285364 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.285533 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.286104 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.286355 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:15:05.286839 29180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.270189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:38133 (local address 127.0.0.1:58624)
0811 09:15:05.273432 (+ 3243us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.273483 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.273701 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.274172 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.274181 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.274816 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.275035 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.275046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.276019 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.276025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.277070 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.277084 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.277906 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.277912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.278099 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.278664 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.278686 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.280714 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.283773 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.283785 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.283794 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.284225 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.284670 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.284673 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.284675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.284812 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.285544 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.285551 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.285752 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.285952 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.286307 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3004,"thread_start_us":2860,"threads_started":1}
I20260811 09:15:05.289230 28252 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.16.125:38133: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e"
member_type: NON_VOTER
last_known_addr {
host: "127.17.16.125"
port: 38133
}
attrs {
promote: true
}
}
dest_uuid: "31eca697b43b43199c0142a966b3b61c"
cas_config_opid_index: -1
I20260811 09:15:05.290750 28252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } }
I20260811 09:15:05.298633 29092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:05.297101 29183 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8f9ed5de1a944954a8c1e5361d98719e (127.17.16.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:05.302649 29092 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Refusing update from remote peer 31eca697b43b43199c0142a966b3b61c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 09:15:05.303864 29184 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:05.307710 29183 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.16.125:38133
I20260811 09:15:05.308254 29183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:05.308463 29183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:05.308730 29022 master.cc:537] Master@127.17.16.125:38133 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.124.keytab.
Entry for principal oryx/127.17.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.124.keytab.
I20260811 09:15:05.322571 29092 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8f9ed5de1a944954a8c1e5361d98719e (127.17.16.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:05.324460 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.124:44323
--webserver_interface=127.17.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.124.keytab
--principal=oryx/127.17.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:39045,127.17.16.125:38133,127.17.16.124:44323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:15:05.347435 29189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:05.347889 29189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: LEARNER
I20260811 09:15:05.348479 29142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31eca697b43b43199c0142a966b3b61c. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:05.348712 29142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: LEARNER
I20260811 09:15:05.348973 29196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:05.349212 29196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: LEARNER
I20260811 09:15:05.350920 29022 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 LEARNER]: Raft consensus shutting down.
I20260811 09:15:05.351483 29022 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 LEARNER]: Raft consensus is shut down!
I20260811 09:15:05.351677 29022 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: stopping tablet replica
I20260811 09:15:05.360100 29022 master.cc:552] Master@127.17.16.125:38133 shutdown complete.
I20260811 09:15:05.360237 29022 master_runner.cc:305] Clearing existing system tablet
I20260811 09:15:05.360704 29022 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:05.360796 29022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:05.363776 29022 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 09:15:05.365427 29198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:05.365806 29022 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:05.366091 29022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 09:15:05.366180 29022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal
uuid: "8f9ed5de1a944954a8c1e5361d98719e"
format_stamp: "Formatted at 2026-08-11 09:15:05 on dist-test-slave-ncp9"
server_key: "41a4938bce315085cf6072a515c4a870"
server_key_iv: "a0fb898be1e6b252f859ee206c345099"
server_key_version: "encryptionkey@0"
I20260811 09:15:05.371138 29022 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:05.378170 29022 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 09:15:05.378350 29022 log.cc:1196] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 09:15:05.378859 29022 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Deleting consensus metadata
I20260811 09:15:05.379128 29022 master_runner.cc:315] Copying system tablet from 127.17.16.126:39045
I20260811 09:15:05.382099 29022 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: tablet copy: Beginning tablet copy session from remote peer at address 127.17.16.126:39045
I20260811 09:15:05.398643 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.383279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50022 (local address 127.17.16.126:39045)
0811 09:15:05.383490 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.383496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.384067 (+ 571us) server_negotiation.cc:407] Connection header received
0811 09:15:05.384324 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.384330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.384422 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.384536 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.385631 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.386781 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.388575 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.389288 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.389413 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.391813 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.391820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.391825 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.391868 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.395484 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.396266 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.396271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.396276 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.396480 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.396970 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.396975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.396979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.397378 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.397531 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.398032 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.398282 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 09:15:05.398679 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.383194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:50022)
0811 09:15:05.383924 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.383945 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.384119 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.384592 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.384597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.385151 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.385448 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.385464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.386948 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.386953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.388408 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.388423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.389483 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.389489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.389671 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.390409 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.390437 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.391598 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.395634 (+ 4036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.395639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.395643 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.396147 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.396609 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.396614 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.396618 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.396821 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.397518 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.397523 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.397723 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.397885 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.398237 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":432,"threads_started":1}
I20260811 09:15:05.400151 28281 tablet_copy_service.cc:140] P 31eca697b43b43199c0142a966b3b61c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8f9ed5de1a944954a8c1e5361d98719e ({username='oryx', principal='oryx/127.17.16.125@KRBTEST.COM'} at 127.0.0.1:50022)
I20260811 09:15:05.400550 28281 tablet_copy_service.cc:161] P 31eca697b43b43199c0142a966b3b61c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8f9ed5de1a944954a8c1e5361d98719e at {username='oryx', principal='oryx/127.17.16.125@KRBTEST.COM'} at 127.0.0.1:50022: session id = 8f9ed5de1a944954a8c1e5361d98719e-00000000000000000000000000000000
I20260811 09:15:05.405565 28281 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 09:15:05.408452 29022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 09:15:05.415616 29022 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: tablet copy: Starting download of 0 data blocks...
I20260811 09:15:05.416023 29022 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: tablet copy: Starting download of 1 WAL segments...
I20260811 09:15:05.420199 29022 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 09:15:05.424722 28281 tablet_copy_service.cc:342] P 31eca697b43b43199c0142a966b3b61c: Request end of tablet copy session 8f9ed5de1a944954a8c1e5361d98719e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.16.125@KRBTEST.COM'} at 127.0.0.1:50022
I20260811 09:15:05.425026 28281 tablet_copy_service.cc:433] P 31eca697b43b43199c0142a966b3b61c: ending tablet copy session 8f9ed5de1a944954a8c1e5361d98719e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8f9ed5de1a944954a8c1e5361d98719e
I20260811 09:15:05.436412 29022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:05.438915 29208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:05.442489 29210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:05.451819 29212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:05.454180 29022 server_base.cc:1034] running on GCE node
I20260811 09:15:05.454584 29022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:05.455230 29022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:05.456470 29022 hybrid_clock.cc:648] HybridClock initialized: now 1786439705456385 us; error 53 us; skew 500 ppm
I20260811 09:15:05.456710 29022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439705, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:05.458976 29022 init.cc:377] Logged in from keytab as oryx/127.17.16.125@KRBTEST.COM (short username oryx)
I20260811 09:15:05.460311 29022 webserver.cc:460] Webserver started at http://127.17.16.125:41019/ using document root <none> and password file <none>
I20260811 09:15:05.460677 29022 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:05.460763 29022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:05.463826 29022 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 09:15:05.465926 29224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:05.466357 29022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 09:15:05.466512 29022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal
uuid: "8f9ed5de1a944954a8c1e5361d98719e"
format_stamp: "Formatted at 2026-08-11 09:15:05 on dist-test-slave-ncp9"
server_key: "41a4938bce315085cf6072a515c4a870"
server_key_iv: "a0fb898be1e6b252f859ee206c345099"
server_key_version: "encryptionkey@0"
I20260811 09:15:05.466733 29022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:05.482582 29022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:05.486562 29022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:05.515710 29022 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.125:38133
I20260811 09:15:05.515841 29313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.125:38133 every 8 connection(s)
I20260811 09:15:05.516669 29022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
I20260811 09:15:05.522641 29315 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:15:05.526701 29315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Bootstrap starting.
I20260811 09:15:05.542336 29315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:15:05.542965 29315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Bootstrap complete.
I20260811 09:15:05.544070 29315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } }
I20260811 09:15:05.544329 29315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8f9ed5de1a944954a8c1e5361d98719e, State: Initialized, Role: LEARNER
I20260811 09:15:05.544638 29315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } }
I20260811 09:15:05.545518 29318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:05.545784 29315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:15:05.545974 29318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: LEARNER
I20260811 09:15:05.553906 29336 catalog_manager.cc:1259] Loaded cluster ID: 363259a4cc274759bc3e778fde34f144
I20260811 09:15:05.554112 29336 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: loading cluster ID for follower catalog manager: success
I20260811 09:15:05.557706 29336 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: acquiring CA information for follower catalog manager: success
I20260811 09:15:05.560199 29336 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:15:05.679967 29013 heartbeater.cc:503] Master 127.17.16.126:39045 was elected leader, sending a full tablet report...
W20260811 09:15:05.715590 29193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:05.715881 29193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:05.716064 29193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:05.723369 29193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:15:05.723480 29193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:05.723555 29193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:05.723632 29193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:15:05.723691 29193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:15:05.734313 29193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:39045,127.17.16.125:38133,127.17.16.124:44323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.124.keytab
--principal=oryx/127.17.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.124:44323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:05.735672 29193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:05.737048 29193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:05.748432 29354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:05.748488 29350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:05.748463 29356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:05.749862 29193 server_base.cc:1034] running on GCE node
I20260811 09:15:05.750645 29193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:05.752038 29193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:05.753237 29193 hybrid_clock.cc:648] HybridClock initialized: now 1786439705753186 us; error 39 us; skew 500 ppm
I20260811 09:15:05.753643 29193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439705, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:05.757005 29193 init.cc:377] Logged in from keytab as oryx/127.17.16.124@KRBTEST.COM (short username oryx)
I20260811 09:15:05.758420 29193 webserver.cc:460] Webserver started at http://127.17.16.124:36941/ using document root <none> and password file <none>
I20260811 09:15:05.759071 29193 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:05.759172 29193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:05.759421 29193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:05.762416 29193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data/instance:
uuid: "76ca417693d045288d11ed58ec291b07"
format_stamp: "Formatted at 2026-08-11 09:15:05 on dist-test-slave-ncp9"
server_key: "d23e212fdcbf97c67eff64bf2718a75a"
server_key_iv: "080814323c53f3d9043aed6c0f93135d"
server_key_version: "encryptionkey@0"
I20260811 09:15:05.763242 29193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal/instance:
uuid: "76ca417693d045288d11ed58ec291b07"
format_stamp: "Formatted at 2026-08-11 09:15:05 on dist-test-slave-ncp9"
server_key: "d23e212fdcbf97c67eff64bf2718a75a"
server_key_iv: "080814323c53f3d9043aed6c0f93135d"
server_key_version: "encryptionkey@0"
I20260811 09:15:05.768435 29193 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:15:05.771915 29367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:05.773370 29193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:05.773573 29193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal
uuid: "76ca417693d045288d11ed58ec291b07"
format_stamp: "Formatted at 2026-08-11 09:15:05 on dist-test-slave-ncp9"
server_key: "d23e212fdcbf97c67eff64bf2718a75a"
server_key_iv: "080814323c53f3d9043aed6c0f93135d"
server_key_version: "encryptionkey@0"
I20260811 09:15:05.773758 29193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:05.795051 29193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:05.799813 29193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:05.800263 29193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:05.821272 29193 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.124:44323
I20260811 09:15:05.821282 29454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.124:44323 every 8 connection(s)
I20260811 09:15:05.822713 29193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
I20260811 09:15:05.824150 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 29193
I20260811 09:15:05.827307 29455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:05.835711 29455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.124@KRBTEST.COM
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439705, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.124@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:05.849974 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.826485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44548 (local address 127.17.16.124:44323)
0811 09:15:05.827252 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.827264 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.827306 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:15:05.828928 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.828954 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.829304 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.829626 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.831603 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.833096 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.835013 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.835636 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.835765 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.841508 (+ 5743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.841519 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.841532 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.841584 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.845162 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.845837 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.845847 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.845852 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.846071 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.846431 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.846437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.846441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.847151 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.847397 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.848028 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.848792 (+ 764us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":353,"threads_started":1}
I20260811 09:15:05.853313 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.838626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:50038)
0811 09:15:05.839408 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.839472 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.839643 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.840040 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.840046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.840853 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.841246 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.841258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.842603 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.842607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.843859 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.843871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.844800 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.844804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.844969 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.845483 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.845499 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.847833 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.850944 (+ 3111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.850955 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.850962 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.851422 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.851794 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.851799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.851802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.851942 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.852468 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.852475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.852635 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.852790 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.853051 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":365,"threads_started":1}
I20260811 09:15:05.853313 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.838910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.17.16.126:39045)
0811 09:15:05.839126 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.839132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.839581 (+ 449us) server_negotiation.cc:407] Connection header received
0811 09:15:05.839774 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.839779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.839849 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.839960 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.841405 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.842443 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.844032 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.844627 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.844770 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.848006 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.848011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.848014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.848049 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.850807 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.851534 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.851539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.851543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.851689 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.852041 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.852045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.852047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.852336 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.852529 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.852867 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.853065 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 09:15:05.854447 29455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439705, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.124@KRBTEST.COM for oryx/127.17.16.125@KRBTEST.COM
I20260811 09:15:05.866067 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.854999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:38133 (local address 127.0.0.1:58628)
0811 09:15:05.855169 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.855185 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.855257 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.855886 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.855888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.856188 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.856318 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.856325 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.857631 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.857634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.858545 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.858555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.859135 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.859139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.859253 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.859700 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.859715 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.861577 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.864168 (+ 2591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.864172 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.864175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.864536 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.864826 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.864829 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.864831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.865000 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.865429 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.865433 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.865561 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.865665 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.865854 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 09:15:05.866154 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.855171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58628 (local address 127.17.16.125:38133)
0811 09:15:05.855651 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.855655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.855671 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:15:05.855719 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.855723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.855783 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.855869 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.856418 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.857521 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.858656 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.859033 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.859128 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.861759 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.861765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.861768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.861799 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.864056 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.864616 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.864619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.864622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.864752 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.865088 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.865091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.865093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.865335 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.865450 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.865752 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.865903 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":242,"threads_started":1}
I20260811 09:15:05.867128 29455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:15:05.870311 29463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:05.870609 29463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439705, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.124@KRBTEST.COM for oryx/127.17.16.124@KRBTEST.COM
I20260811 09:15:05.879748 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.870937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.17.16.125:38133)
0811 09:15:05.873259 (+ 2322us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.873263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.873669 (+ 406us) server_negotiation.cc:407] Connection header received
0811 09:15:05.873813 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.873818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.873918 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.874033 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:05.874496 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.875509 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.877161 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.878452 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.878597 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.878780 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.879383 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.879522 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2209}
I20260811 09:15:05.879916 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.870447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:38133 (local address 127.0.0.1:58632)
0811 09:15:05.873563 (+ 3116us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.873578 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.873699 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.874107 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.874111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.874143 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:05.874355 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.874366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.875680 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.875684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.876974 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.876988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.878880 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.878884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.879150 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.879253 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.879342 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.879452 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3019,"thread_start_us":2923,"threads_started":1}
I20260811 09:15:05.881024 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.867827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:44323 (local address 127.0.0.1:44556)
0811 09:15:05.868013 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.868029 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.868118 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.868389 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.868392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.868802 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.868975 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.868986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.870044 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.870049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.871325 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.871342 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.872408 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.872414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.872628 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.873227 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.873247 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.875493 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.878570 (+ 3077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.878573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.878576 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.878939 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.879338 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.879343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.879346 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.879527 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.880121 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.880126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.880290 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.880398 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.880618 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 09:15:05.881008 29318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31eca697b43b43199c0142a966b3b61c. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:05.881248 29318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: LEARNER
I20260811 09:15:05.881096 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.868020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.17.16.124:44323)
0811 09:15:05.868160 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.868165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.868180 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:15:05.868217 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.868221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.868267 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.868349 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.869103 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.869906 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.871488 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.872127 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.872233 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.875649 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.875654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.875657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.875684 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.878471 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.879064 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.879070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.879074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.879230 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.879648 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.879652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.879655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.879998 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.880142 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.880528 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.880745 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":44}
I20260811 09:15:05.882035 29465 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c: attempting to promote NON_VOTER 8f9ed5de1a944954a8c1e5361d98719e to VOTER
I20260811 09:15:05.882880 29465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } }
I20260811 09:15:05.884562 29283 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 LEARNER]: Refusing update from remote peer 31eca697b43b43199c0142a966b3b61c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 09:15:05.885078 29464 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 09:15:05.886571 29455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: Bootstrap starting.
I20260811 09:15:05.888058 29465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8f9ed5de1a944954a8c1e5361d98719e (127.17.16.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } }
I20260811 09:15:05.888712 29283 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 8f9ed5de1a944954a8c1e5361d98719e (127.17.16.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } }
I20260811 09:15:05.890566 29455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:05.890758 29465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } }
I20260811 09:15:05.890970 29465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:05.891265 29463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } }
I20260811 09:15:05.891418 29463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:05.891947 29455 log.cc:824] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:05.892289 29318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } }
I20260811 09:15:05.892516 29318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:15:05.894680 29455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: No bootstrap required, opened a new log
I20260811 09:15:05.902081 29455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } }
I20260811 09:15:05.902397 29455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:05.902494 29455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ca417693d045288d11ed58ec291b07, State: Initialized, Role: FOLLOWER
I20260811 09:15:05.903064 29455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } }
I20260811 09:15:05.904024 29472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } } }
I20260811 09:15:05.904343 29472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:15:05.905104 29455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:15:05.913883 29486 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:15:05.914024 29486 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:15:05.917994 29193 master_runner.cc:428] Detected that this master 76ca417693d045288d11ed58ec291b07 is joining an existing cluster
I20260811 09:15:05.918092 29193 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.16.124:44323
I20260811 09:15:05.939441 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.921881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50046 (local address 127.17.16.126:39045)
0811 09:15:05.922062 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.922068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.922854 (+ 786us) server_negotiation.cc:407] Connection header received
0811 09:15:05.923106 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.923111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.923196 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.923270 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.924832 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.926060 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.928208 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.928910 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.929015 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.932414 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.932422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.932427 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.932469 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.936051 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.936841 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.936846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.936850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.937029 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.937555 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.937560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.937563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.937920 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.938041 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.938847 (+ 806us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.939085 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 09:15:05.940867 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.921975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:50046)
0811 09:15:05.922761 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.922779 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.922951 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.923482 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.923486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.924233 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.924645 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.924662 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.926259 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.926265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.928025 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.928042 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.929419 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.929427 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.929891 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.930910 (+ 1019us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.930936 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.932180 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.936224 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.936229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.936237 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.936714 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.937183 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.937188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.937192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.937394 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.938140 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.938145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.938502 (+ 357us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.939986 (+ 1484us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.940314 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":87,"thread_start_us":458,"threads_started":1}
I20260811 09:15:05.950779 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.921881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44570 (local address 127.17.16.124:44323)
0811 09:15:05.922058 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.922063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.934784 (+ 12721us) server_negotiation.cc:407] Connection header received
0811 09:15:05.935153 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.935159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.935229 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.935287 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.936603 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.937685 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.939775 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.940429 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.940526 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.942717 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.942723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.942727 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.942757 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.946883 (+ 4126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.948478 (+ 1595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.948484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.948488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.948637 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.949090 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.949096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.949099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.949572 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.949743 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.950202 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.950420 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":58}
I20260811 09:15:05.951784 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.922401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:44323 (local address 127.0.0.1:44570)
0811 09:15:05.934682 (+ 12281us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.934707 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.935018 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.935462 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.935467 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.936126 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.936464 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.936476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.937883 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.937888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.939625 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.939658 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.940934 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.940940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.941121 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.941715 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.941737 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.942547 (+ 810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.947853 (+ 5306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.947857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.947861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.948362 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.948771 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.948775 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.948778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.948969 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.949724 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.949729 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.949913 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.950999 (+ 1086us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.951346 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12115,"mutex_wait_us":23,"thread_start_us":15336,"threads_started":1}
I20260811 09:15:05.954705 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.923337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58644 (local address 127.17.16.125:38133)
0811 09:15:05.923512 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.923517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.937719 (+ 14202us) server_negotiation.cc:407] Connection header received
0811 09:15:05.938230 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.938236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.938322 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.938392 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.939882 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.941045 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.943885 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.944476 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.944568 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.947421 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.947428 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.947433 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.947473 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.950859 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.951649 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.951655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.951658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.951804 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.953139 (+ 1335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.953144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.953147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.953491 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.953652 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.954143 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.954369 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:15:05.954751 29497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.922861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:38133 (local address 127.0.0.1:58644)
0811 09:15:05.937596 (+ 14735us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.937614 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.938058 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.938597 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.938602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.939283 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.939692 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.939705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.941241 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.941247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.943713 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.943730 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.944823 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.944827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.944998 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.945573 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.945596 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.947249 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.951002 (+ 3753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.951007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.951011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.951502 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.952067 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.952073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.952076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.952971 (+ 895us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.953616 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.953620 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.953850 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.953997 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.954293 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14602,"thread_start_us":11943,"threads_started":1}
I20260811 09:15:05.956854 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 28358
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439703, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.126@KRBTEST.COM for oryx/127.17.16.124@KRBTEST.COM
I20260811 09:15:05.964483 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.946966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:44323 (local address 127.0.0.1:44572)
0811 09:15:05.947541 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:05.947560 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:05.947660 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:05.948321 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:05.948327 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:05.948785 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:05.949010 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.949024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.951477 (+ 2453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.951483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.952777 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:05.952792 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.953719 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.953725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:05.953894 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.955580 (+ 1686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:05.955602 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:05.957919 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:05.962025 (+ 4106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.962031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.962035 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.962537 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.962905 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:05.962910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:05.962914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:05.963034 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:05.963584 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:05.963588 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:05.963735 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 09:15:05.963854 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 09:15:05.964117 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458}
I20260811 09:15:05.964550 29502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.947167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44572 (local address 127.17.16.124:44323)
0811 09:15:05.947899 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.947908 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.947931 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:15:05.948024 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.948030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.948110 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.949189 (+ 1079us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.950027 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.951314 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.953004 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.953546 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.957081 (+ 3535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.958147 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.958155 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.958160 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.958195 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.961884 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.962665 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.962669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.962671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.962793 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.963153 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.963157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.963159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.963467 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.963595 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.963992 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.964178 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":521,"thread_start_us":554,"threads_started":1}
I20260811 09:15:05.965919 28251 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.16.124:44323: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "76ca417693d045288d11ed58ec291b07"
member_type: NON_VOTER
last_known_addr {
host: "127.17.16.124"
port: 44323
}
attrs {
promote: true
}
}
dest_uuid: "31eca697b43b43199c0142a966b3b61c"
cas_config_opid_index: 6
I20260811 09:15:05.966773 28251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } }
I20260811 09:15:05.969506 29283 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Refusing update from remote peer 31eca697b43b43199c0142a966b3b61c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
Aug 11 09:15:05 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.65@KRBTEST.COM
I20260811 09:15:05.970172 29411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:05.973675 29411 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 FOLLOWER]: Refusing update from remote peer 31eca697b43b43199c0142a966b3b61c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 09:15:05.974332 29465 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 09:15:05.975257 29465 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:05.977365 29465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 76ca417693d045288d11ed58ec291b07 (127.17.16.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } } }
I20260811 09:15:05.977893 29283 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 76ca417693d045288d11ed58ec291b07 (127.17.16.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } } }
I20260811 09:15:05.979926 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:05.958560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.17.16.65:45197)
0811 09:15:05.959273 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:05.959280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:05.959305 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:15:05.959360 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:05.959365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:05.959564 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:05.959738 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:05.961111 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.962219 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.964035 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:05.965360 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:05.966054 (+ 694us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:05.971394 (+ 5340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:05.971411 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:05.971431 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:05.971475 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:05.975398 (+ 3923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.976313 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.976322 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.976325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.976489 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:05.977059 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:05.977063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:05.977066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:05.977663 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:05.979069 (+ 1406us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:05.979237 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 09:15:05.979517 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":406,"threads_started":1}
I20260811 09:15:05.982264 29318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } } }
I20260811 09:15:05.982538 29318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:15:05.985562 28566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:05.987039 29465 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.16.124:44323
I20260811 09:15:05.987645 29193 master.cc:537] Master@127.17.16.124:44323 shutting down...
I20260811 09:15:05.987459 29464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } } }
I20260811 09:15:05.987737 29464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:05.988277 29465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } } }
I20260811 09:15:05.988523 29465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:05.990300 29410 raft_consensus.cc:858] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } }
W20260811 09:15:05.990487 29410 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } }
I20260811 09:15:06.011405 29410 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8f9ed5de1a944954a8c1e5361d98719e (127.17.16.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:06.028805 29518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:06.029368 29518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 09:15:06.029775 29472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31eca697b43b43199c0142a966b3b61c. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:06.029991 29472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 09:15:06.030239 29512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: NON_VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: true } } }
I20260811 09:15:06.030622 29512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 09:15:06.031466 29463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } } }
I20260811 09:15:06.031790 29463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
W20260811 09:15:06.032199 28226 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 09:15:06.033463 29193 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 09:15:06.033931 29193 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 09:15:06.034116 29193 tablet_replica.cc:330] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: stopping tablet replica
W20260811 09:15:06.035475 28226 consensus_peers.cc:487] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c -> Peer 76ca417693d045288d11ed58ec291b07 (127.17.16.124:44323): Couldn't send request to peer 76ca417693d045288d11ed58ec291b07. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 09:15:06.052963 29193 master.cc:552] Master@127.17.16.124:44323 shutdown complete.
I20260811 09:15:06.053076 29193 master_runner.cc:305] Clearing existing system tablet
I20260811 09:15:06.053469 29193 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:06.053586 29193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:06.073876 29193 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.007s sys 0.011s
I20260811 09:15:06.083629 29521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:06.084093 29193 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:06.084367 29193 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.004s
I20260811 09:15:06.084481 29193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal
uuid: "76ca417693d045288d11ed58ec291b07"
format_stamp: "Formatted at 2026-08-11 09:15:05 on dist-test-slave-ncp9"
server_key: "d23e212fdcbf97c67eff64bf2718a75a"
server_key_iv: "080814323c53f3d9043aed6c0f93135d"
server_key_version: "encryptionkey@0"
I20260811 09:15:06.089354 29193 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:06.096333 29193 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 09:15:06.096486 29193 log.cc:1196] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 09:15:06.097144 29193 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: Deleting consensus metadata
I20260811 09:15:06.097414 29193 master_runner.cc:315] Copying system tablet from 127.17.16.126:39045
I20260811 09:15:06.100333 29193 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: tablet copy: Beginning tablet copy session from remote peer at address 127.17.16.126:39045
I20260811 09:15:06.114552 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 28358
I20260811 09:15:06.129309 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.101211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.17.16.126:39045)
0811 09:15:06.101445 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:06.101451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:06.105627 (+ 4176us) server_negotiation.cc:407] Connection header received
0811 09:15:06.105949 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:06.105956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:06.106063 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:06.106174 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:06.107300 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.108406 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.110294 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.110947 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.111071 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.114316 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:06.114323 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:06.114328 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:06.114368 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:06.126142 (+ 11774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:06.127059 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:06.127065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:06.127069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:06.127235 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:06.127667 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:06.127673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:06.127676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:06.128047 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:06.128200 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:06.128680 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 09:15:06.128974 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 09:15:06.129370 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.101207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:50056)
0811 09:15:06.105472 (+ 4265us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:06.105492 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:06.105757 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:06.106221 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:06.106225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:06.106818 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:06.107097 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:06.107111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.108573 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.108579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:06.110134 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:06.110152 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.111128 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.111135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:06.111327 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.112692 (+ 1365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:06.112715 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:06.114091 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:06.126327 (+ 12236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:06.126333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:06.126338 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:06.126913 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:06.127363 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:06.127368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:06.127371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:06.127544 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:06.128169 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:06.128174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:06.128386 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 09:15:06.128550 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 09:15:06.128882 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4115,"thread_start_us":3905,"threads_started":1}
I20260811 09:15:06.129954 28281 tablet_copy_service.cc:140] P 31eca697b43b43199c0142a966b3b61c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 76ca417693d045288d11ed58ec291b07 ({username='oryx', principal='oryx/127.17.16.124@KRBTEST.COM'} at 127.0.0.1:50056)
I20260811 09:15:06.130141 28281 tablet_copy_service.cc:161] P 31eca697b43b43199c0142a966b3b61c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 76ca417693d045288d11ed58ec291b07 at {username='oryx', principal='oryx/127.17.16.124@KRBTEST.COM'} at 127.0.0.1:50056: session id = 76ca417693d045288d11ed58ec291b07-00000000000000000000000000000000
I20260811 09:15:06.133507 28281 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 09:15:06.135414 29193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 09:15:06.142753 29193 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: tablet copy: Starting download of 0 data blocks...
I20260811 09:15:06.143064 29193 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: tablet copy: Starting download of 1 WAL segments...
I20260811 09:15:06.145288 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:45197
--local_ip_for_outbound_sockets=127.17.16.65
--tserver_master_addrs=127.17.16.126:39045
--webserver_port=36051
--webserver_interface=127.17.16.65
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab
--principal=oryx/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.16.126:39045,127.17.16.125:38133
--tserver_master_addrs=127.17.16.126:39045,127.17.16.125:38133,127.17.16.124:44323 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:15:06.145406 29193 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 09:15:06.148589 28281 tablet_copy_service.cc:342] P 31eca697b43b43199c0142a966b3b61c: Request end of tablet copy session 76ca417693d045288d11ed58ec291b07-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.16.124@KRBTEST.COM'} at 127.0.0.1:50056
I20260811 09:15:06.148710 28281 tablet_copy_service.cc:433] P 31eca697b43b43199c0142a966b3b61c: ending tablet copy session 76ca417693d045288d11ed58ec291b07-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 76ca417693d045288d11ed58ec291b07
I20260811 09:15:06.164577 29193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:06.168654 29532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:06.169543 29534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:06.173350 29538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:06.174027 29193 server_base.cc:1034] running on GCE node
I20260811 09:15:06.174410 29193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:06.175118 29193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:06.176330 29193 hybrid_clock.cc:648] HybridClock initialized: now 1786439706176280 us; error 33 us; skew 500 ppm
I20260811 09:15:06.176573 29193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:06 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439706, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:06.178813 29193 init.cc:377] Logged in from keytab as oryx/127.17.16.124@KRBTEST.COM (short username oryx)
I20260811 09:15:06.181911 29193 webserver.cc:460] Webserver started at http://127.17.16.124:35155/ using document root <none> and password file <none>
I20260811 09:15:06.182366 29193 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:06.182536 29193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:06.185741 29193 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 09:15:06.187695 29547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:06.188109 29193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 09:15:06.188269 29193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal
uuid: "76ca417693d045288d11ed58ec291b07"
format_stamp: "Formatted at 2026-08-11 09:15:05 on dist-test-slave-ncp9"
server_key: "d23e212fdcbf97c67eff64bf2718a75a"
server_key_iv: "080814323c53f3d9043aed6c0f93135d"
server_key_version: "encryptionkey@0"
I20260811 09:15:06.188516 29193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:06.229238 29193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:06.233575 29193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:06.261348 29193 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.124:44323
I20260811 09:15:06.261439 29636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.124:44323 every 8 connection(s)
I20260811 09:15:06.262574 29193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
I20260811 09:15:06.268208 29639 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:15:06.272156 29639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: Bootstrap starting.
I20260811 09:15:06.287925 29639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:15:06.288511 29639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: Bootstrap complete.
I20260811 09:15:06.289819 29639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } }
I20260811 09:15:06.290119 29639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 76ca417693d045288d11ed58ec291b07, State: Initialized, Role: LEARNER
I20260811 09:15:06.290381 29639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } }
I20260811 09:15:06.291744 29639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:15:06.291419 29643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } } }
I20260811 09:15:06.292014 29643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: This master's current role is: LEARNER
I20260811 09:15:06.300346 29664 catalog_manager.cc:1259] Loaded cluster ID: 363259a4cc274759bc3e778fde34f144
I20260811 09:15:06.300448 29664 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: loading cluster ID for follower catalog manager: success
I20260811 09:15:06.303938 29664 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: acquiring CA information for follower catalog manager: success
I20260811 09:15:06.306311 29664 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:15:06.503822 29464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } } }
I20260811 09:15:06.504149 29464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:06.510102 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.504311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44582 (local address 127.17.16.124:44323)
0811 09:15:06.505044 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:06.505049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:06.505064 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:15:06.505126 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:06.505130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:06.505198 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:06.505306 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:06.505770 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.506731 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.508310 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.509153 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.509274 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.509514 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:06.509666 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 09:15:06.509797 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":413,"threads_started":1}
I20260811 09:15:06.510124 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.504200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:44323 (local address 127.0.0.1:44582)
0811 09:15:06.504666 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:06.504689 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:06.504855 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:06.505352 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:06.505356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:06.505384 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:06.505627 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:06.505637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.506884 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.506890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:06.508150 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:06.508164 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.509318 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.509324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:06.509515 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.509549 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:15:06.509638 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 09:15:06.509781 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":302,"threads_started":1}
I20260811 09:15:06.510989 29600 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260811 09:15:06.511353 29643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31eca697b43b43199c0142a966b3b61c. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: NON_VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: true } } }
I20260811 09:15:06.511632 29643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: This master's current role is: LEARNER
I20260811 09:15:06.511936 29464 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c: attempting to promote NON_VOTER 76ca417693d045288d11ed58ec291b07 to VOTER
I20260811 09:15:06.512569 29464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } }
I20260811 09:15:06.514282 29600 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 LEARNER]: Refusing update from remote peer 31eca697b43b43199c0142a966b3b61c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 09:15:06.514295 29283 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Refusing update from remote peer 31eca697b43b43199c0142a966b3b61c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 09:15:06.514861 29463 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 09:15:06.516876 29465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 76ca417693d045288d11ed58ec291b07 (127.17.16.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:06.517550 29283 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 76ca417693d045288d11ed58ec291b07 (127.17.16.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:06.520668 29465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:06.520987 29465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:06.521302 29465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:06.521579 29465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:06.529343 29464 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260811 09:15:06.534088 29531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:06.534519 29531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:06.534672 29531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 09:15:06.545395 29670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:06.545763 29670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:15:06.549269 29600 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 76ca417693d045288d11ed58ec291b07 (127.17.16.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
W20260811 09:15:06.551306 29531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:06.551422 29531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:06.551571 29531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:15:06.552460 29643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:06.552651 29643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:15:06.563113 29531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab
--principal=oryx/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:45197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=36051
--tserver_master_addrs=127.17.16.126:39045,127.17.16.125:38133,127.17.16.124:44323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:06.564988 29531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:06.566648 29531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:06.585952 29678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:06.586107 29679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:06.586405 29683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:06.587812 29531 server_base.cc:1034] running on GCE node
I20260811 09:15:06.588600 29531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:06.589856 29531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:06.591075 29531 hybrid_clock.cc:648] HybridClock initialized: now 1786439706590963 us; error 103 us; skew 500 ppm
I20260811 09:15:06.591387 29531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:06 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439706, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:06.594830 29531 init.cc:377] Logged in from keytab as oryx/127.17.16.65@KRBTEST.COM (short username oryx)
I20260811 09:15:06.596292 29531 webserver.cc:460] Webserver started at http://127.17.16.65:36051/ using document root <none> and password file <none>
I20260811 09:15:06.597174 29531 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:06.597383 29531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:06.603287 29531 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260811 09:15:06.606454 29699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:06.607752 29531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:15:06.607942 29531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "be3e7b06847640edaf6a5a221d39a9f2"
format_stamp: "Formatted at 2026-08-11 09:15:03 on dist-test-slave-ncp9"
server_key: "7f2682dadbae68d0cd074b5edc7d39c7"
server_key_iv: "53d5def81eed6e0a5cca7a106f4fd0b4"
server_key_version: "encryptionkey@0"
I20260811 09:15:06.608973 29531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:06.627360 29531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:06.631155 29531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:06.631381 29531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:06.632360 29531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:06.634119 29531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:06.634209 29531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:06.634291 29531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:06.634344 29531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:06.666330 29531 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:45197
I20260811 09:15:06.666400 29879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:45197 every 8 connection(s)
I20260811 09:15:06.667826 29531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:15:06.677793 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 29531
I20260811 09:15:06.678092 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 28591
Aug 11 09:15:06 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439706, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.124@KRBTEST.COM
Aug 11 09:15:06 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439706, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.125@KRBTEST.COM
Aug 11 09:15:06 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.66@KRBTEST.COM
I20260811 09:15:06.711899 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.670515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:57483 (local address 127.17.16.124:44323)
0811 09:15:06.670737 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:06.670744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:06.686444 (+ 15700us) server_negotiation.cc:407] Connection header received
0811 09:15:06.687448 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:06.687455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:06.687547 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:06.687617 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:06.690300 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.691472 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.697043 (+ 5571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.697708 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.697806 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.703705 (+ 5899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:06.703712 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:06.703717 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:06.703758 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:06.707937 (+ 4179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:06.709007 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:06.709014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:06.709018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:06.709193 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:06.709738 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:06.709743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:06.709746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:06.710120 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:06.710246 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:06.711311 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0811 09:15:06.711544 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 09:15:06 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439706, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:06.713568 29890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.670624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:39541 (local address 127.17.16.125:38133)
0811 09:15:06.677374 (+ 6750us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:06.677380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:06.684473 (+ 7093us) server_negotiation.cc:407] Connection header received
0811 09:15:06.687214 (+ 2741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:06.687221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:06.687330 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:06.687456 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:06.697252 (+ 9796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.698327 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.700275 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.700925 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.701030 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.705702 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:06.705710 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:06.705714 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:06.705759 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:06.709403 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:06.710617 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:06.710622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:06.710625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:06.710787 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:06.711426 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:06.711432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:06.711435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:06.711849 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:06.712286 (+ 437us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:06.712867 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 09:15:06.713196 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6591,"thread_start_us":6459,"threads_started":1}
I20260811 09:15:06.714517 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.671151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:44323 (local address 127.17.16.65:57483)
0811 09:15:06.686336 (+ 15185us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:06.686353 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:06.687281 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:06.687993 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:06.688006 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:06.689017 (+ 1011us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:06.690061 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:06.690090 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.691770 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.691776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:06.696819 (+ 5043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:06.696834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.698176 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.698183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:06.698465 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.699388 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:06.699422 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:06.703418 (+ 3996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:06.708132 (+ 4714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:06.708143 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:06.708163 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:06.708850 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:06.709345 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:06.709350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:06.709354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:06.709606 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:06.710336 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:06.710344 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:06.710805 (+ 461us) client_negotiation.cc:769] Sending connection context
0811 09:15:06.712305 (+ 1500us) client_negotiation.cc:240] Negotiation successful
0811 09:15:06.712861 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15035,"mutex_wait_us":34,"spinlock_wait_cycles":69376,"thread_start_us":13608,"threads_started":1}
I20260811 09:15:06.715313 29893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.671151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:38133 (local address 127.17.16.65:39541)
0811 09:15:06.682544 (+ 11393us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:06.682612 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:06.686909 (+ 4297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:06.687996 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:06.688007 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:06.696534 (+ 8527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:06.697024 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:06.697039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.698521 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.698526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:06.700095 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:06.700112 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.701426 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.701434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:06.701819 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.702695 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:06.702717 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:06.705436 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:06.709828 (+ 4392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:06.709833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:06.709836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:06.710462 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:06.710922 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:06.710926 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:06.710929 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:06.711266 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:06.712375 (+ 1109us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:06.712380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:06.712546 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 09:15:06.714136 (+ 1590us) client_negotiation.cc:240] Negotiation successful
0811 09:15:06.714441 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11042,"mutex_wait_us":77,"thread_start_us":14730,"threads_started":1}
I20260811 09:15:06.716064 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.679067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47206 (local address 127.17.16.66:39355)
0811 09:15:06.694153 (+ 15086us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:06.694163 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:06.694194 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:15:06.694265 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:06.694273 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:06.694554 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:06.694705 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:06.696260 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.697699 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.699580 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.700469 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.700604 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.707807 (+ 7203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:06.707829 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:06.707854 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:06.707903 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:06.711927 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:06.712823 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:06.712833 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:06.712837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:06.713031 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:06.713481 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:06.713485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:06.713489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:06.713909 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:06.714272 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:06.715412 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0811 09:15:06.715683 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14885,"thread_start_us":14716,"threads_started":1}
I20260811 09:15:06.716598 29881 heartbeater.cc:347] Connected to a master server at 127.17.16.124:44323
I20260811 09:15:06.716796 29885 heartbeater.cc:347] Connected to a master server at 127.17.16.125:38133
I20260811 09:15:06.716954 29885 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:06.716957 29881 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:06.717414 28784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:06.717885 29881 heartbeater.cc:511] Master 127.17.16.124:44323 requested a full tablet report, sending...
I20260811 09:15:06.718714 29885 heartbeater.cc:511] Master 127.17.16.125:38133 requested a full tablet report, sending...
I20260811 09:15:06.720996 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.692804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.65:42065)
0811 09:15:06.701466 (+ 8662us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:06.701493 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:06.701626 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:06.704341 (+ 2715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:06.704345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:06.705010 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:06.705385 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:06.705398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.707115 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.707121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:06.708652 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:06.708668 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.710029 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.710036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:06.710246 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.711018 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:06.711038 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:06.713788 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:06.718066 (+ 4278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:06.718071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:06.718075 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:06.718649 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:06.719120 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:06.719124 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:06.719128 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:06.719299 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:06.720023 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:06.720028 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:06.720178 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 09:15:06.720325 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 09:15:06.720613 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8421,"thread_start_us":8324,"threads_started":1}
I20260811 09:15:06.721312 29584 ts_manager.cc:194] Registered new tserver with Master: be3e7b06847640edaf6a5a221d39a9f2 (127.17.16.65:45197)
I20260811 09:15:06.721740 29897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:06.686605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:42065 (local address 127.17.16.126:39045)
0811 09:15:06.703856 (+ 17251us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:06.703863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:06.703883 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:15:06.703944 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:06.703949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:06.704023 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:06.704127 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:06.705610 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.706848 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.708843 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:06.709613 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:06.709728 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:06.714039 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:06.714046 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:06.714050 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:06.714091 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:06.717909 (+ 3818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:06.718807 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:06.718813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:06.718816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:06.718975 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:06.719429 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:06.719434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:06.719437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:06.719813 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:06.719948 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:06.720473 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 09:15:06.720719 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17004,"thread_start_us":17138,"threads_started":1}
I20260811 09:15:06.722643 29256 ts_manager.cc:194] Registered new tserver with Master: be3e7b06847640edaf6a5a221d39a9f2 (127.17.16.65:45197)
I20260811 09:15:06.723645 29884 heartbeater.cc:347] Connected to a master server at 127.17.16.126:39045
I20260811 09:15:06.723755 29884 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:06.724112 29884 heartbeater.cc:511] Master 127.17.16.126:39045 requested a full tablet report, sending...
I20260811 09:15:06.725332 28251 ts_manager.cc:194] Re-registered known tserver with Master: be3e7b06847640edaf6a5a221d39a9f2 (127.17.16.65:45197)
I20260811 09:15:06.726469 28251 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:42065
I20260811 09:15:06.800283 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 28591
I20260811 09:15:06.823395 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:39355
--local_ip_for_outbound_sockets=127.17.16.66
--tserver_master_addrs=127.17.16.126:39045
--webserver_port=35379
--webserver_interface=127.17.16.66
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab
--principal=oryx/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.16.126:39045,127.17.16.125:38133
--tserver_master_addrs=127.17.16.126:39045,127.17.16.125:38133,127.17.16.124:44323 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:15:07.066764 29669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31eca697b43b43199c0142a966b3b61c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:07.067129 29669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c [sys.catalog]: This master's current role is: LEADER
W20260811 09:15:07.155294 29904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:07.155563 29904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:07.155701 29904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:07.162612 29904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:07.162706 29904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:07.162850 29904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:15:07.172093 29904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab
--principal=oryx/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:39355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=35379
--tserver_master_addrs=127.17.16.126:39045,127.17.16.125:38133,127.17.16.124:44323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:07.173418 29904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:07.174763 29904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:07.187580 29916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:07.187518 29918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:07.187588 29920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:07.188175 29904 server_base.cc:1034] running on GCE node
I20260811 09:15:07.188802 29904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:07.189751 29904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:07.190961 29904 hybrid_clock.cc:648] HybridClock initialized: now 1786439707190907 us; error 42 us; skew 500 ppm
I20260811 09:15:07.191253 29904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:07 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439707, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:07.194444 29904 init.cc:377] Logged in from keytab as oryx/127.17.16.66@KRBTEST.COM (short username oryx)
I20260811 09:15:07.195869 29904 webserver.cc:460] Webserver started at http://127.17.16.66:35379/ using document root <none> and password file <none>
I20260811 09:15:07.196488 29904 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:07.196626 29904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:07.201822 29904 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:15:07.204470 29930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:07.205808 29904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:15:07.205976 29904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "dde75c9995e94b5cbbe56c74a12a9445"
format_stamp: "Formatted at 2026-08-11 09:15:04 on dist-test-slave-ncp9"
server_key: "25d88c56ed0dbd18f4ed11b28475c660"
server_key_iv: "61c451ae1bebe10e3b22d1739b1fc5f1"
server_key_version: "encryptionkey@0"
I20260811 09:15:07.206889 29904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:07.222793 29904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:07.226933 29904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:07.227371 29904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:07.228361 29904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:07.229825 29904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:07.229925 29904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:07.230032 29904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:07.230111 29904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:07.258998 29904 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:39355
I20260811 09:15:07.259068 30094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:39355 every 8 connection(s)
I20260811 09:15:07.260442 29904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:15:07.267359 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 29904
I20260811 09:15:07.267645 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 28802
Aug 11 09:15:07 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.67@KRBTEST.COM
Aug 11 09:15:07 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439707, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
Aug 11 09:15:07 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439707, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.124@KRBTEST.COM
Aug 11 09:15:07 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439707, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.125@KRBTEST.COM
I20260811 09:15:07.329317 30101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.264338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:46803 (local address 127.17.16.124:44323)
0811 09:15:07.305567 (+ 41229us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.305574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.310032 (+ 4458us) server_negotiation.cc:407] Connection header received
0811 09:15:07.310111 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.310117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.310202 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.310311 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:07.312819 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.313919 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.316072 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.316697 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.316811 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.322013 (+ 5202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:07.322020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:07.322025 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:07.322064 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:07.325086 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.326037 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.326043 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.326047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.326207 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.327129 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.327135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.327138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.327512 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:07.327657 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.328605 (+ 948us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.328867 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41077,"thread_start_us":40963,"threads_started":1}
I20260811 09:15:07.330785 30103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.265187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:48771 (local address 127.17.16.126:39045)
0811 09:15:07.301301 (+ 36114us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.301309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.307601 (+ 6292us) server_negotiation.cc:407] Connection header received
0811 09:15:07.309694 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.309699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.309800 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.310023 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:07.312409 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.313510 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.316759 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.317284 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.317395 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.321561 (+ 4166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:07.321566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:07.321569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:07.321602 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:07.325500 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.326578 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.326584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.326588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.326746 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.327188 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.327194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.327197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.327545 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:07.327680 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.330170 (+ 2490us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.330402 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35953,"thread_start_us":42428,"threads_started":1}
I20260811 09:15:07.331029 30102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.263958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.66:48771)
0811 09:15:07.306297 (+ 42339us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:07.306363 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:07.309395 (+ 3032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:07.310298 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:07.310310 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:07.311263 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:07.312226 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.312258 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.313658 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.313665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.315902 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.315921 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.317491 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.317497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.317683 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.318223 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:07.318247 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:07.321291 (+ 3044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:07.326057 (+ 4766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.326061 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.326065 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.326465 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.326849 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.326852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.326854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.327083 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.329713 (+ 2630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:07.329717 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:07.329900 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 09:15:07.330050 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 09:15:07.330337 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41947,"mutex_wait_us":10,"spinlock_wait_cycles":18816,"thread_start_us":44616,"threads_started":1}
I20260811 09:15:07.331539 30105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.268731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36008 (local address 127.17.16.67:40501)
0811 09:15:07.294507 (+ 25776us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.294518 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.302812 (+ 8294us) server_negotiation.cc:407] Connection header received
0811 09:15:07.302908 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.302917 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.303269 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.303435 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:07.308650 (+ 5215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.311000 (+ 2350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.312821 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.313599 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.313724 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.321544 (+ 7820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:07.321560 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:07.321576 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:07.321625 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:07.327500 (+ 5875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.328225 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.328232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.328235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.328366 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.328812 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.328818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.328821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.329288 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:07.329592 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.330814 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.331096 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25539,"thread_start_us":25810,"threads_started":1}
I20260811 09:15:07.331532 30100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.263959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:44323 (local address 127.17.16.66:46803)
0811 09:15:07.309056 (+ 45097us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:07.309084 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:07.309751 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:07.310419 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:07.310423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:07.311532 (+ 1109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:07.312642 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.312656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.314070 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.314075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.315891 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.315920 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.316952 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.316971 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.317241 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.317956 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:07.318003 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:07.321808 (+ 3805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:07.325245 (+ 3437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.325254 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.325271 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.325915 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.326349 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.326353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.326358 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.327011 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.327739 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:07.327747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:07.328185 (+ 438us) client_negotiation.cc:769] Sending connection context
0811 09:15:07.328517 (+ 332us) client_negotiation.cc:240] Negotiation successful
0811 09:15:07.329087 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44901,"mutex_wait_us":685,"thread_start_us":41219,"threads_started":1}
I20260811 09:15:07.332669 30098 heartbeater.cc:347] Connected to a master server at 127.17.16.126:39045
I20260811 09:15:07.332978 29002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:07.333029 30109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.309408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:55629 (local address 127.17.16.125:38133)
0811 09:15:07.311084 (+ 1676us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.311090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.313722 (+ 2632us) server_negotiation.cc:407] Connection header received
0811 09:15:07.313952 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.313958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.314065 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.314218 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:07.315593 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.316609 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.319383 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.319989 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.320121 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.325360 (+ 5239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:07.325366 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:07.325370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:07.325409 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:07.328952 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.329714 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.329719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.329723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.329881 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.331343 (+ 1462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.331348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.331350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.331713 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:07.331856 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.332398 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.332626 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1544,"thread_start_us":1423,"threads_started":1}
I20260811 09:15:07.333042 30098 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:07.333725 30095 heartbeater.cc:347] Connected to a master server at 127.17.16.124:44323
I20260811 09:15:07.333868 30095 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:07.334028 30098 heartbeater.cc:511] Master 127.17.16.126:39045 requested a full tablet report, sending...
I20260811 09:15:07.334314 30095 heartbeater.cc:511] Master 127.17.16.124:44323 requested a full tablet report, sending...
I20260811 09:15:07.334939 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.311076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:38133 (local address 127.17.16.66:55629)
0811 09:15:07.312336 (+ 1260us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:07.313653 (+ 1317us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:07.313818 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:07.314435 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:07.314439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:07.315042 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:07.315407 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.315420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.317621 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.317626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.319212 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.319227 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.320162 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.320167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.320324 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.320886 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:07.320929 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:07.325159 (+ 4230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:07.329105 (+ 3946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.329109 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.329112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.329582 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.331023 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.331029 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.331033 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.331203 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.331939 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:07.331943 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:07.332112 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 09:15:07.332956 (+ 844us) client_negotiation.cc:240] Negotiation successful
0811 09:15:07.333480 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1146,"thread_start_us":1015,"threads_started":1}
I20260811 09:15:07.335777 28251 ts_manager.cc:194] Re-registered known tserver with Master: dde75c9995e94b5cbbe56c74a12a9445 (127.17.16.66:39355)
I20260811 09:15:07.336061 29584 ts_manager.cc:194] Registered new tserver with Master: dde75c9995e94b5cbbe56c74a12a9445 (127.17.16.66:39355)
I20260811 09:15:07.336714 28251 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:48771
I20260811 09:15:07.339079 30099 heartbeater.cc:347] Connected to a master server at 127.17.16.125:38133
I20260811 09:15:07.339236 30099 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:07.339658 30099 heartbeater.cc:511] Master 127.17.16.125:38133 requested a full tablet report, sending...
I20260811 09:15:07.340806 29256 ts_manager.cc:194] Registered new tserver with Master: dde75c9995e94b5cbbe56c74a12a9445 (127.17.16.66:39355)
I20260811 09:15:07.420790 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 28802
I20260811 09:15:07.445436 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:40501
--local_ip_for_outbound_sockets=127.17.16.67
--tserver_master_addrs=127.17.16.126:39045
--webserver_port=44631
--webserver_interface=127.17.16.67
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab
--principal=oryx/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.16.126:39045,127.17.16.125:38133
--tserver_master_addrs=127.17.16.126:39045,127.17.16.125:38133,127.17.16.124:44323 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:15:07.728658 29884 heartbeater.cc:503] Master 127.17.16.126:39045 was elected leader, sending a full tablet report...
W20260811 09:15:07.772825 30115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:07.773195 30115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:07.773360 30115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:07.780767 30115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:07.780932 30115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:07.781121 30115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:15:07.791118 30115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab
--principal=oryx/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:40501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=44631
--tserver_master_addrs=127.17.16.126:39045,127.17.16.125:38133,127.17.16.124:44323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:07.792495 30115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:07.793911 30115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:07.806746 30126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:07.806792 30121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:07.806816 30124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:07.807966 30115 server_base.cc:1034] running on GCE node
I20260811 09:15:07.808528 30115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:07.809509 30115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:07.810720 30115 hybrid_clock.cc:648] HybridClock initialized: now 1786439707810666 us; error 46 us; skew 500 ppm
I20260811 09:15:07.811034 30115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:07 dist-test-slave-ncp9 krb5kdc[28177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439707, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:07.814260 30115 init.cc:377] Logged in from keytab as oryx/127.17.16.67@KRBTEST.COM (short username oryx)
I20260811 09:15:07.815631 30115 webserver.cc:460] Webserver started at http://127.17.16.67:44631/ using document root <none> and password file <none>
I20260811 09:15:07.816318 30115 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:07.816452 30115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:07.821919 30115 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:15:07.824626 30137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:07.826083 30115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:07.826243 30115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "9170330144184e8385e0338b4b2521c2"
format_stamp: "Formatted at 2026-08-11 09:15:04 on dist-test-slave-ncp9"
server_key: "6ec055dd7e406c9b368b569c123315c0"
server_key_iv: "e7da8a08ae624f8b36851452d02993d2"
server_key_version: "encryptionkey@0"
I20260811 09:15:07.827260 30115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:07.845788 30115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:07.849974 30115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:07.850248 30115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:07.851197 30115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:07.852572 30115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:07.852669 30115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:07.852772 30115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:07.852841 30115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:07.883822 30115 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:40501
I20260811 09:15:07.883888 30324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:40501 every 8 connection(s)
I20260811 09:15:07.885304 30115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:15:07.888800 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 30115
Aug 11 09:15:07 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439707, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.125@KRBTEST.COM
Aug 11 09:15:07 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439707, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.124@KRBTEST.COM
Aug 11 09:15:07 dist-test-slave-ncp9 krb5kdc[28177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439707, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:07.930075 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.888740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:44323 (local address 127.17.16.67:33847)
0811 09:15:07.902845 (+ 14105us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:07.902865 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:07.906505 (+ 3640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:07.907167 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:07.907173 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:07.907841 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:07.908203 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.908215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.914039 (+ 5824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.914045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.915695 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.915710 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.916845 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.916851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.917059 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.917714 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:07.917736 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:07.920335 (+ 2599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:07.925138 (+ 4803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.925144 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.925148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.925720 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.926144 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.926149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.926153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.926386 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.927196 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:07.927206 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:07.927661 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 09:15:07.927974 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 09:15:07.928514 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13962,"mutex_wait_us":3,"spinlock_wait_cycles":70016,"thread_start_us":20567,"threads_started":1}
I20260811 09:15:07.931542 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.888005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:33847 (local address 127.17.16.124:44323)
0811 09:15:07.888500 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.888507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.902969 (+ 14462us) server_negotiation.cc:407] Connection header received
0811 09:15:07.906720 (+ 3751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.906728 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.906826 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.906909 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:07.912738 (+ 5829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.913859 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.915867 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.916451 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.916576 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.920575 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:07.920582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:07.920587 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:07.920630 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:07.924980 (+ 4350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.925861 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.925868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.925872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.926020 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.926511 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.926517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.926520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.926976 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:07.927118 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.930873 (+ 3755us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.931125 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":440,"threads_started":1}
I20260811 09:15:07.933492 30325 heartbeater.cc:347] Connected to a master server at 127.17.16.124:44323
I20260811 09:15:07.933861 30325 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:07.934512 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.888322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:36087 (local address 127.17.16.125:38133)
0811 09:15:07.893312 (+ 4990us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.893320 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.900431 (+ 7111us) server_negotiation.cc:407] Connection header received
0811 09:15:07.903762 (+ 3331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.903770 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.903881 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.903991 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:07.908413 (+ 4422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.909687 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.913774 (+ 4087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.914456 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.914561 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.920087 (+ 5526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:07.920093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:07.920098 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:07.920148 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:07.923743 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.924777 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.924784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.924788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.924974 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.932732 (+ 7758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.932738 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.932741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.933149 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:07.933279 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.933914 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.934164 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4849,"thread_start_us":4747,"threads_started":1}
I20260811 09:15:07.934881 30325 heartbeater.cc:511] Master 127.17.16.124:44323 requested a full tablet report, sending...
I20260811 09:15:07.935532 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.888535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:38133 (local address 127.17.16.67:36087)
0811 09:15:07.900199 (+ 11664us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:07.900264 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:07.903268 (+ 3004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:07.904476 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:07.904491 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:07.905417 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:07.906329 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.906355 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.909938 (+ 3583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.909944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.912202 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.912220 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.914927 (+ 2707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.914934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.915768 (+ 834us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.916498 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:07.916525 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:07.919804 (+ 3279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:07.923987 (+ 4183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.923998 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.924020 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.924628 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.925149 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.925154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.925158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.932577 (+ 7419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.933380 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:07.933384 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:07.933609 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 09:15:07.934782 (+ 1173us) client_negotiation.cc:240] Negotiation successful
0811 09:15:07.935095 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11249,"mutex_wait_us":364,"thread_start_us":11813,"threads_started":1}
I20260811 09:15:07.936343 30330 heartbeater.cc:347] Connected to a master server at 127.17.16.125:38133
I20260811 09:15:07.936486 30330 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:07.936942 30330 heartbeater.cc:511] Master 127.17.16.125:38133 requested a full tablet report, sending...
I20260811 09:15:07.937958 29584 ts_manager.cc:194] Registered new tserver with Master: 9170330144184e8385e0338b4b2521c2 (127.17.16.67:40501)
I20260811 09:15:07.938086 29256 ts_manager.cc:194] Registered new tserver with Master: 9170330144184e8385e0338b4b2521c2 (127.17.16.67:40501)
I20260811 09:15:07.948112 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.901957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:59165 (local address 127.17.16.126:39045)
0811 09:15:07.913121 (+ 11164us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.913125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.913631 (+ 506us) server_negotiation.cc:407] Connection header received
0811 09:15:07.913744 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.913748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.913842 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.913943 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:07.915385 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.916816 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.919362 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.919957 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.920042 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.940137 (+ 20095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:07.940144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:07.940150 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:07.940193 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:07.944338 (+ 4145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.945357 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.945363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.945367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.945674 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.946253 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.946259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.946263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.946691 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:07.946831 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.947465 (+ 634us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.947738 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11048,"thread_start_us":10939,"threads_started":1}
I20260811 09:15:07.948426 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.901849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.67:59165)
0811 09:15:07.913497 (+ 11648us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:07.913512 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:07.913627 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:07.914126 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:07.914131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:07.914765 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:07.915199 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.915213 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.917007 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.917012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.919198 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.919213 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.920388 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.920397 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.920587 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.921281 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:07.921304 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:07.939907 (+ 18603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:07.944507 (+ 4600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.944513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.944517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.945214 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.945835 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:07.945841 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:07.945845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:07.946120 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:07.946916 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:07.946920 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:07.947165 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 09:15:07.947625 (+ 460us) client_negotiation.cc:240] Negotiation successful
0811 09:15:07.947982 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10792,"mutex_wait_us":3,"thread_start_us":10606,"threads_started":1}
I20260811 09:15:07.949472 30326 heartbeater.cc:347] Connected to a master server at 127.17.16.126:39045
I20260811 09:15:07.949643 30326 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:07.950246 30326 heartbeater.cc:511] Master 127.17.16.126:39045 requested a full tablet report, sending...
I20260811 09:15:07.951308 28251 ts_manager.cc:194] Re-registered known tserver with Master: 9170330144184e8385e0338b4b2521c2 (127.17.16.67:40501)
I20260811 09:15:07.952535 28251 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:59165
I20260811 09:15:07.955952 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.918483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.17.16.125:38133)
0811 09:15:07.930023 (+ 11540us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.930040 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.931494 (+ 1454us) server_negotiation.cc:407] Connection header received
0811 09:15:07.931564 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.931570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.932022 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.932155 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:07.936679 (+ 4524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.937975 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.940210 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.940949 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.941079 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.945070 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:07.945076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:07.945082 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:07.945118 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:07.949148 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.951288 (+ 2140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.951294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.951297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.951532 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.951991 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.951996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.951999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.952332 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:07.952531 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.955356 (+ 2825us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.955568 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11395,"thread_start_us":11228,"threads_started":1}
I20260811 09:15:07.957698 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.917749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.17.16.126:39045)
0811 09:15:07.934331 (+ 16582us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.934336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.934352 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:15:07.934400 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.934403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.934462 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.934567 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:07.936735 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.937975 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.940284 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.941013 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.941722 (+ 709us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.944838 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:07.944844 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:07.944849 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:07.944886 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:07.950021 (+ 5135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.950847 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.950852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.950855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.951001 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.951472 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.951476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.951479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.951882 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:07.957118 (+ 5236us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.957219 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.957378 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":16472,"thread_start_us":16351,"threads_started":1}
I20260811 09:15:07.967203 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.930913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44586 (local address 127.17.16.124:44323)
0811 09:15:07.932183 (+ 1270us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.932188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.952929 (+ 20741us) server_negotiation.cc:407] Connection header received
0811 09:15:07.953154 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.953160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.953237 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.953309 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:07.955309 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.956343 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.958143 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.958718 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.958848 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.961587 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:07.961594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:07.961599 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:07.961637 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:07.964723 (+ 3086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.965353 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.965359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.965362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.965515 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:07.965801 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:07.965805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:07.965809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:07.966143 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:07.966403 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.966621 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.966850 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1175,"thread_start_us":25467,"threads_started":1}
I20260811 09:15:07.961609 28251 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:15:07.970646 28251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:15:07.997910 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.987582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:40501 (local address 127.0.0.1:36016)
0811 09:15:07.988531 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:07.988568 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:07.988766 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:07.989576 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:07.989581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:07.989606 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:07.990370 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.990387 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.992496 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.992502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.994997 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.995010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.996669 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.996676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.996838 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.996873 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:15:07.997357 (+ 484us) client_negotiation.cc:240] Negotiation successful
0811 09:15:07.997540 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":379,"threads_started":1}
I20260811 09:15:07.998590 30377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.987867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36016 (local address 127.17.16.67:40501)
0811 09:15:07.988800 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.988809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.988855 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:15:07.988987 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.988995 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.989279 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.989547 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:07.990679 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.992334 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.995201 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.996437 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.996565 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.996823 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.998201 (+ 1378us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.998353 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":591,"threads_started":1}
I20260811 09:15:07.999876 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.987910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:39355 (local address 127.0.0.1:47218)
0811 09:15:07.989087 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:07.989109 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:07.989278 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:07.990679 (+ 1401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:07.990684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:07.990708 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:07.991425 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.991441 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.993260 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.993265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.997346 (+ 4081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:07.997367 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.999344 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.999350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:07.999481 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.999502 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 09:15:07.999573 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 09:15:07.999661 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"thread_start_us":336,"threads_started":1}
I20260811 09:15:08.000105 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.988350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.17.16.66:39355)
0811 09:15:07.990072 (+ 1722us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.990080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:07.990109 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:15:07.990215 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:07.990222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:07.990445 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:07.991668 (+ 1223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:07.991871 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.993107 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.997618 (+ 4511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:07.998838 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:07.998919 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:07.999058 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:07.999653 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 09:15:07.999824 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1564,"thread_start_us":2598,"threads_started":1}
I20260811 09:15:08.003135 30229 tablet_service.cc:1467] Processing CreateTablet for tablet fedbbdee535c4c2ab9929b0a3c4b7def (DEFAULT_TABLE table=test-table [id=77b1a422e4814b86bfbc309803d84aaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:08.003628 29998 tablet_service.cc:1467] Processing CreateTablet for tablet fedbbdee535c4c2ab9929b0a3c4b7def (DEFAULT_TABLE table=test-table [id=77b1a422e4814b86bfbc309803d84aaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:08.004665 30229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fedbbdee535c4c2ab9929b0a3c4b7def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:08.004725 29998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fedbbdee535c4c2ab9929b0a3c4b7def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:08.007804 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.990054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45197 (local address 127.0.0.1:34258)
0811 09:15:07.997740 (+ 7686us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:07.997760 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:07.997923 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:08.000776 (+ 2853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:08.000780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:08.000806 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:08.001176 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:08.001189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.003036 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.003041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:08.004992 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:08.005009 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.007031 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.007037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:08.007223 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.007257 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:15:08.007346 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 09:15:08.007474 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7490,"thread_start_us":7144,"threads_started":1}
I20260811 09:15:08.008076 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:07.990507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.17.16.65:45197)
0811 09:15:07.991049 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:07.991057 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:08.000185 (+ 9128us) server_negotiation.cc:407] Connection header received
0811 09:15:08.000301 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:08.000308 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:08.000560 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:08.000774 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:08.001375 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.002866 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.005193 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.006830 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.006966 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.007156 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:08.007418 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 09:15:08.007598 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":256,"threads_started":1}
I20260811 09:15:08.012969 29784 tablet_service.cc:1467] Processing CreateTablet for tablet fedbbdee535c4c2ab9929b0a3c4b7def (DEFAULT_TABLE table=test-table [id=77b1a422e4814b86bfbc309803d84aaa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:08.014694 29784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fedbbdee535c4c2ab9929b0a3c4b7def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:08.017827 30387 tablet_bootstrap.cc:492] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: Bootstrap starting.
I20260811 09:15:08.020257 30389 tablet_bootstrap.cc:492] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: Bootstrap starting.
I20260811 09:15:08.021720 30387 tablet_bootstrap.cc:654] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:08.022724 30387 log.cc:824] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:08.023887 30389 tablet_bootstrap.cc:654] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:08.025158 30389 log.cc:824] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:08.026355 30387 tablet_bootstrap.cc:492] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: No bootstrap required, opened a new log
I20260811 09:15:08.026625 30387 ts_tablet_manager.cc:1397] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260811 09:15:08.028105 30389 tablet_bootstrap.cc:492] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: No bootstrap required, opened a new log
I20260811 09:15:08.028365 30389 ts_tablet_manager.cc:1397] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 09:15:08.031085 30397 tablet_bootstrap.cc:492] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: Bootstrap starting.
I20260811 09:15:08.034518 30397 tablet_bootstrap.cc:654] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:08.035787 30397 log.cc:824] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:08.036193 30387 raft_consensus.cc:348] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } } peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } }
I20260811 09:15:08.036630 30387 raft_consensus.cc:374] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:08.036728 30387 raft_consensus.cc:729] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9170330144184e8385e0338b4b2521c2, State: Initialized, Role: FOLLOWER
I20260811 09:15:08.036484 30389 raft_consensus.cc:348] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } } peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } }
I20260811 09:15:08.036940 30389 raft_consensus.cc:374] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:08.037046 30389 raft_consensus.cc:729] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dde75c9995e94b5cbbe56c74a12a9445, State: Initialized, Role: FOLLOWER
I20260811 09:15:08.037351 30387 consensus_queue.cc:260] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } } peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } }
I20260811 09:15:08.037644 30389 consensus_queue.cc:260] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } } peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } }
I20260811 09:15:08.038875 30397 tablet_bootstrap.cc:492] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: No bootstrap required, opened a new log
I20260811 09:15:08.039073 30387 ts_tablet_manager.cc:1428] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260811 09:15:08.039170 30397 ts_tablet_manager.cc:1397] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:15:08.039332 30326 heartbeater.cc:503] Master 127.17.16.126:39045 was elected leader, sending a full tablet report...
I20260811 09:15:08.043886 30098 heartbeater.cc:503] Master 127.17.16.126:39045 was elected leader, sending a full tablet report...
I20260811 09:15:08.046449 30389 ts_tablet_manager.cc:1428] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: Time spent starting tablet: real 0.018s user 0.003s sys 0.013s
I20260811 09:15:08.051407 30397 raft_consensus.cc:348] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } } peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } }
I20260811 09:15:08.051797 30397 raft_consensus.cc:374] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:08.051898 30397 raft_consensus.cc:729] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be3e7b06847640edaf6a5a221d39a9f2, State: Initialized, Role: FOLLOWER
I20260811 09:15:08.052485 30397 consensus_queue.cc:260] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } } peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } }
I20260811 09:15:08.054250 30397 ts_tablet_manager.cc:1428] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
W20260811 09:15:08.058928 30107 tablet.cc:2369] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:15:08.137573 30331 tablet.cc:2369] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:15:08.177544 29887 tablet.cc:2369] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:15:08.245383 30403 raft_consensus.cc:482] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:15:08.245784 30403 raft_consensus.cc:504] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } } peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } }
I20260811 09:15:08.247602 30403 leader_election.cc:288] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9170330144184e8385e0338b4b2521c2 (127.17.16.67:40501), be3e7b06847640edaf6a5a221d39a9f2 (127.17.16.65:45197)
I20260811 09:15:08.254351 30377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:08.247985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:52505 (local address 127.17.16.67:40501)
0811 09:15:08.248194 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:08.248200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:08.248707 (+ 507us) server_negotiation.cc:407] Connection header received
0811 09:15:08.248940 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:08.248946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:08.249015 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:08.249093 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:08.249685 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.250404 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.252696 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.253496 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.253658 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.253738 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:08.253984 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 09:15:08.254110 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:15:08.254514 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:08.247815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:40501 (local address 127.17.16.66:52505)
0811 09:15:08.248575 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:08.248591 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:08.248762 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:08.249129 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:08.249134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:08.249219 (+ 85us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:08.249534 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:08.249544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.250544 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.250549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:08.252528 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:08.252545 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.253680 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.253686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:08.253834 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.253858 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:15:08.253944 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 09:15:08.254081 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":65,"spinlock_wait_cycles":49024,"thread_start_us":419,"threads_started":1}
I20260811 09:15:08.254792 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:08.248140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:52415 (local address 127.17.16.65:45197)
0811 09:15:08.248395 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:08.248403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:08.249148 (+ 745us) server_negotiation.cc:407] Connection header received
0811 09:15:08.249390 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:08.249394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:08.249465 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:08.249559 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:08.250189 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.250964 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.252993 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.253818 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.253998 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.254082 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:08.254381 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 09:15:08.254518 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 09:15:08.254931 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:08.247933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45197 (local address 127.17.16.66:52415)
0811 09:15:08.249008 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:08.249025 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:08.249174 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:08.249608 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:08.249613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:08.249639 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:08.249999 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:08.250009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.251115 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.251121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:08.252782 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:08.252797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.254049 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.254056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:08.254208 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.254239 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:15:08.254324 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 09:15:08.254478 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":23,"thread_start_us":674,"threads_started":1}
I20260811 09:15:08.255698 30261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedbbdee535c4c2ab9929b0a3c4b7def" candidate_uuid: "dde75c9995e94b5cbbe56c74a12a9445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9170330144184e8385e0338b4b2521c2" is_pre_election: true
I20260811 09:15:08.255906 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedbbdee535c4c2ab9929b0a3c4b7def" candidate_uuid: "dde75c9995e94b5cbbe56c74a12a9445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be3e7b06847640edaf6a5a221d39a9f2" is_pre_election: true
I20260811 09:15:08.256291 30261 raft_consensus.cc:2454] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dde75c9995e94b5cbbe56c74a12a9445 in term 0.
I20260811 09:15:08.256415 29813 raft_consensus.cc:2454] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dde75c9995e94b5cbbe56c74a12a9445 in term 0.
I20260811 09:15:08.256960 29933 leader_election.cc:302] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9170330144184e8385e0338b4b2521c2, dde75c9995e94b5cbbe56c74a12a9445; no voters:
I20260811 09:15:08.257411 30403 raft_consensus.cc:2781] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:15:08.257558 30403 raft_consensus.cc:482] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:15:08.257658 30403 raft_consensus.cc:3037] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:08.260136 30403 raft_consensus.cc:504] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } } peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } }
I20260811 09:15:08.260860 30403 leader_election.cc:288] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [CANDIDATE]: Term 1 election: Requested vote from peers 9170330144184e8385e0338b4b2521c2 (127.17.16.67:40501), be3e7b06847640edaf6a5a221d39a9f2 (127.17.16.65:45197)
I20260811 09:15:08.261286 30261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedbbdee535c4c2ab9929b0a3c4b7def" candidate_uuid: "dde75c9995e94b5cbbe56c74a12a9445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9170330144184e8385e0338b4b2521c2"
I20260811 09:15:08.261507 30261 raft_consensus.cc:3037] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:08.261711 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fedbbdee535c4c2ab9929b0a3c4b7def" candidate_uuid: "dde75c9995e94b5cbbe56c74a12a9445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be3e7b06847640edaf6a5a221d39a9f2"
I20260811 09:15:08.261900 29813 raft_consensus.cc:3037] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:08.264670 30261 raft_consensus.cc:2454] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dde75c9995e94b5cbbe56c74a12a9445 in term 1.
I20260811 09:15:08.265064 29933 leader_election.cc:302] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9170330144184e8385e0338b4b2521c2, dde75c9995e94b5cbbe56c74a12a9445; no voters:
I20260811 09:15:08.265388 30403 raft_consensus.cc:2781] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:08.265360 29813 raft_consensus.cc:2454] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dde75c9995e94b5cbbe56c74a12a9445 in term 1.
I20260811 09:15:08.266306 30403 raft_consensus.cc:686] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 1 LEADER]: Becoming Leader. State: Replica: dde75c9995e94b5cbbe56c74a12a9445, State: Running, Role: LEADER
I20260811 09:15:08.266814 30403 consensus_queue.cc:237] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } } peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } }
I20260811 09:15:08.276443 28252 catalog_manager.cc:5495] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 reported cstate change: term changed from 0 to 1, leader changed from <none> to dde75c9995e94b5cbbe56c74a12a9445 (127.17.16.66). New cstate: current_term: 1 leader_uuid: "dde75c9995e94b5cbbe56c74a12a9445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } health_report { overall_health: UNKNOWN } } }
I20260811 09:15:08.363936 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:08.358082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.17.16.66:39355)
0811 09:15:08.358302 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:08.358308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:08.358328 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:15:08.358456 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:08.358460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:08.358539 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:08.358670 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:15:08.359053 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.359967 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.361529 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.362160 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.362289 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.362724 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:15:08.363239 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:15:08.363393 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:08.363486 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 09:15:08.363569 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:15:08.372031 29813 raft_consensus.cc:1261] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2 [term 1 FOLLOWER]: Refusing update from remote peer dde75c9995e94b5cbbe56c74a12a9445: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:08.372176 30261 raft_consensus.cc:1261] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2 [term 1 FOLLOWER]: Refusing update from remote peer dde75c9995e94b5cbbe56c74a12a9445: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:08.373075 30409 consensus_queue.cc:1035] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:08.373399 30403 consensus_queue.cc:1035] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:08.402055 28252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:08.402367 28252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:08.408514 28252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31eca697b43b43199c0142a966b3b61c: Sending DeleteTablet for 3 replicas of tablet fedbbdee535c4c2ab9929b0a3c4b7def
I20260811 09:15:08.409581 30229 tablet_service.cc:1514] Processing DeleteTablet for tablet fedbbdee535c4c2ab9929b0a3c4b7def with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:08 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:36016
I20260811 09:15:08.410526 30430 tablet_replica.cc:330] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: stopping tablet replica
I20260811 09:15:08.411123 30430 raft_consensus.cc:2229] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:08.411490 29998 tablet_service.cc:1514] Processing DeleteTablet for tablet fedbbdee535c4c2ab9929b0a3c4b7def with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:08 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:47218
I20260811 09:15:08.411543 29784 tablet_service.cc:1514] Processing DeleteTablet for tablet fedbbdee535c4c2ab9929b0a3c4b7def with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:08 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:34258
I20260811 09:15:08.411901 30430 raft_consensus.cc:2258] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:08.412784 30431 tablet_replica.cc:330] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: stopping tablet replica
I20260811 09:15:08.413343 30431 raft_consensus.cc:2229] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:15:08.414072 30431 raft_consensus.cc:2258] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:08.414676 30433 tablet_replica.cc:330] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: stopping tablet replica
I20260811 09:15:08.415280 30433 raft_consensus.cc:2229] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:08.415807 30433 raft_consensus.cc:2258] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:08.417397 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 28200
I20260811 09:15:08.418560 30433 ts_tablet_manager.cc:1905] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:08.418867 28301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:08.419907 30430 ts_tablet_manager.cc:1905] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:08.425573 30431 ts_tablet_manager.cc:1905] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:08.426889 30430 ts_tablet_manager.cc:1918] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:08.427125 30430 log.cc:1196] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/fedbbdee535c4c2ab9929b0a3c4b7def
I20260811 09:15:08.427693 30430 ts_tablet_manager.cc:1939] T fedbbdee535c4c2ab9929b0a3c4b7def P 9170330144184e8385e0338b4b2521c2: Deleting consensus metadata
I20260811 09:15:08.430150 30433 ts_tablet_manager.cc:1918] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:08.430349 30433 log.cc:1196] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/fedbbdee535c4c2ab9929b0a3c4b7def
I20260811 09:15:08.430920 30433 ts_tablet_manager.cc:1939] T fedbbdee535c4c2ab9929b0a3c4b7def P be3e7b06847640edaf6a5a221d39a9f2: Deleting consensus metadata
I20260811 09:15:08.437546 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:08.414842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58654 (local address 127.17.16.125:38133)
0811 09:15:08.415020 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:08.415026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:08.416851 (+ 1825us) server_negotiation.cc:407] Connection header received
0811 09:15:08.421518 (+ 4667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:08.421524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:08.421607 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:08.421687 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:08.425014 (+ 3327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.426097 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.427931 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.428516 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.428615 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.431738 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:08.431743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:08.431746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:08.431778 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:08.434575 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:08.435613 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:08.435619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:08.435623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:08.435747 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:08.436110 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:08.436115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:08.436119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:08.436390 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:08.436609 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:08.437004 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 09:15:08.437226 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:15:08.439742 30431 ts_tablet_manager.cc:1918] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:08.439919 30431 log.cc:1196] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/fedbbdee535c4c2ab9929b0a3c4b7def
I20260811 09:15:08.440557 30431 ts_tablet_manager.cc:1939] T fedbbdee535c4c2ab9929b0a3c4b7def P dde75c9995e94b5cbbe56c74a12a9445: Deleting consensus metadata
I20260811 09:15:08.452505 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:08.416761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44596 (local address 127.17.16.124:44323)
0811 09:15:08.416952 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:08.416958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:08.437930 (+ 20972us) server_negotiation.cc:407] Connection header received
0811 09:15:08.438195 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:08.438201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:08.438287 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:08.438355 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:08.440026 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.440790 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.442187 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.442656 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.442781 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.445954 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:08.445962 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:08.445967 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:08.446012 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:08.449494 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:08.450324 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:08.450330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:08.450334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:08.450507 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:08.450914 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:08.450919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:08.450923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:08.451313 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:08.451567 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:08.451897 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 09:15:08.452144 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 09:15:08.568543 28227 catalog_manager.cc:4841] TS 9170330144184e8385e0338b4b2521c2 (127.17.16.67:40501): tablet fedbbdee535c4c2ab9929b0a3c4b7def (table test-table [id=77b1a422e4814b86bfbc309803d84aaa]) successfully deleted
I20260811 09:15:08.569176 28229 catalog_manager.cc:4841] TS be3e7b06847640edaf6a5a221d39a9f2 (127.17.16.65:45197): tablet fedbbdee535c4c2ab9929b0a3c4b7def (table test-table [id=77b1a422e4814b86bfbc309803d84aaa]) successfully deleted
I20260811 09:15:08.569149 28226 catalog_manager.cc:4841] TS dde75c9995e94b5cbbe56c74a12a9445 (127.17.16.66:39355): tablet fedbbdee535c4c2ab9929b0a3c4b7def (table test-table [id=77b1a422e4814b86bfbc309803d84aaa]) successfully deleted
W20260811 09:15:08.568967 28227 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.17.16.67:40501, user_credentials={real_user=slave}} blocked reactor thread for 139305us
W20260811 09:15:08.569391 28229 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.17.16.65:45197, user_credentials={real_user=slave}} blocked reactor thread for 136497us
I20260811 09:15:08.569799 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 28200
I20260811 09:15:08.631130 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:08.610263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44608 (local address 127.17.16.124:44323)
0811 09:15:08.610541 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:08.610548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:08.612549 (+ 2001us) server_negotiation.cc:407] Connection header received
0811 09:15:08.612833 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:08.612841 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:08.612950 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:08.613044 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:08.615707 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.616953 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.619131 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.619822 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.619932 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.623974 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:08.623981 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:08.623986 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:08.624029 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:08.627565 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:08.628705 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:08.628711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:08.628715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:08.628875 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:08.629340 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:08.629345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:08.629348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:08.629716 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:08.630005 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:08.630383 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 09:15:08.630715 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260811 09:15:08.635764 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:08.611464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.17.16.125:38133)
0811 09:15:08.611676 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:08.611682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:08.620726 (+ 9044us) server_negotiation.cc:407] Connection header received
0811 09:15:08.621301 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:08.621307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:08.621401 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:08.621475 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:08.623559 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.624648 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.626545 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:08.627200 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:08.627298 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:08.630437 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:08.630443 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:08.630448 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:08.630487 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:08.633294 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:08.633900 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:08.633906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:08.633909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:08.634040 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:08.634387 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:08.634391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:08.634393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:08.634689 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:08.634906 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:08.635232 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 09:15:08.635440 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:15:09.394210 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.393473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.66:47689)
0811 09:15:09.393883 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.394017 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":213,"threads_started":1}
W20260811 09:15:09.394629 30098 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:39045 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
I20260811 09:15:09.399916 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.399045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.67:59391)
0811 09:15:09.399624 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.399754 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":247,"threads_started":1}
W20260811 09:15:09.400286 30326 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:39045 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
I20260811 09:15:09.411514 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.410785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.65:36743)
0811 09:15:09.411224 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.411341 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":186,"threads_started":1}
W20260811 09:15:09.411885 29884 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:39045 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
I20260811 09:15:09.909183 30449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31eca697b43b43199c0142a966b3b61c)
I20260811 09:15:09.909474 30449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } }
I20260811 09:15:09.909752 30451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 31eca697b43b43199c0142a966b3b61c)
I20260811 09:15:09.910013 30451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } }
I20260811 09:15:09.911005 30449 leader_election.cc:288] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31eca697b43b43199c0142a966b3b61c (127.17.16.126:39045), 76ca417693d045288d11ed58ec291b07 (127.17.16.124:44323)
I20260811 09:15:09.912416 30453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.911181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:44242)
0811 09:15:09.911950 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.912109 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":520,"threads_started":1}
W20260811 09:15:09.912796 29230 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
I20260811 09:15:09.913774 30451 leader_election.cc:288] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31eca697b43b43199c0142a966b3b61c (127.17.16.126:39045), 8f9ed5de1a944954a8c1e5361d98719e (127.17.16.125:38133)
I20260811 09:15:09.914618 30453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.914203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:44256)
0811 09:15:09.914397 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.914479 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 09:15:09.914745 30456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.911332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:44254)
0811 09:15:09.914258 (+ 2926us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.914456 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2797,"thread_start_us":2684,"threads_started":1}
W20260811 09:15:09.914943 29230 leader_election.cc:334] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 31eca697b43b43199c0142a966b3b61c (127.17.16.126:39045): Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
W20260811 09:15:09.915621 29550 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
I20260811 09:15:09.917821 30461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.917310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:44272)
0811 09:15:09.917470 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.917632 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 09:15:09.918236 29550 leader_election.cc:334] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 31eca697b43b43199c0142a966b3b61c (127.17.16.126:39045): Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
I20260811 09:15:09.921115 30457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.911723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.17.16.124:44323)
0811 09:15:09.912391 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:09.912399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:09.912933 (+ 534us) server_negotiation.cc:407] Connection header received
0811 09:15:09.913263 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:09.913271 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:09.913352 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:09.913421 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:09.914304 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:09.915551 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:09.918735 (+ 3184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:09.920013 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:09.920104 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:09.920206 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:09.920688 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 09:15:09.920795 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":340,"threads_started":1}
I20260811 09:15:09.921159 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.911304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:44323 (local address 127.0.0.1:41976)
0811 09:15:09.912803 (+ 1499us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.912820 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:09.913060 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:09.913687 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:09.913691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:09.913735 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:09.914074 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:09.914089 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:09.915729 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:09.915734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:09.918551 (+ 2817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:09.918568 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:09.920345 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:09.920351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:09.920521 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:09.920551 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 09:15:09.920650 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 09:15:09.920777 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1397,"mutex_wait_us":5,"thread_start_us":1225,"threads_started":1}
I20260811 09:15:09.921677 29600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f9ed5de1a944954a8c1e5361d98719e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "76ca417693d045288d11ed58ec291b07" is_pre_election: true
I20260811 09:15:09.921955 29600 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f9ed5de1a944954a8c1e5361d98719e in term 1.
I20260811 09:15:09.922433 29229 leader_election.cc:302] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76ca417693d045288d11ed58ec291b07, 8f9ed5de1a944954a8c1e5361d98719e; no voters: 31eca697b43b43199c0142a966b3b61c
I20260811 09:15:09.922922 30459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.914460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53850 (local address 127.17.16.125:38133)
0811 09:15:09.915954 (+ 1494us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:09.915958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:09.915978 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:15:09.916020 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:09.916023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:09.916089 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:09.916169 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:09.916824 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:09.918733 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:09.920979 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:09.921974 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:09.922117 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:09.922224 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:09.922558 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 09:15:09.922675 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1386,"thread_start_us":1530,"threads_started":1}
I20260811 09:15:09.922927 30456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.914766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:38133 (local address 127.0.0.1:53850)
0811 09:15:09.914937 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.914952 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:09.915069 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:09.916212 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:09.916218 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:09.916262 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:09.916610 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:09.916623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:09.918909 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:09.918914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:09.920792 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:09.920808 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:09.922168 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:09.922175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:09.922382 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:09.922417 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:15:09.922513 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 09:15:09.922654 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":618,"threads_started":1}
I20260811 09:15:09.923084 30449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 09:15:09.923213 30449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 31eca697b43b43199c0142a966b3b61c)
I20260811 09:15:09.923333 30449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:15:09.923655 29283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "76ca417693d045288d11ed58ec291b07" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8f9ed5de1a944954a8c1e5361d98719e" is_pre_election: true
I20260811 09:15:09.925595 30449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } }
I20260811 09:15:09.926111 29283 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76ca417693d045288d11ed58ec291b07 in current term 2: Already voted for candidate 8f9ed5de1a944954a8c1e5361d98719e in this term.
I20260811 09:15:09.926182 30449 leader_election.cc:288] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [CANDIDATE]: Term 2 election: Requested vote from peers 31eca697b43b43199c0142a966b3b61c (127.17.16.126:39045), 76ca417693d045288d11ed58ec291b07 (127.17.16.124:44323)
I20260811 09:15:09.926589 29600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f9ed5de1a944954a8c1e5361d98719e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "76ca417693d045288d11ed58ec291b07"
I20260811 09:15:09.926787 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.926419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:44288)
0811 09:15:09.926575 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.926657 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:15:09.926836 29600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:15:09.927707 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.927328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:44294)
0811 09:15:09.927487 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.927595 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 09:15:09.927888 29230 leader_election.cc:334] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 31eca697b43b43199c0142a966b3b61c (127.17.16.126:39045): Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
I20260811 09:15:09.929153 29600 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f9ed5de1a944954a8c1e5361d98719e in term 2.
I20260811 09:15:09.929374 29550 leader_election.cc:302] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76ca417693d045288d11ed58ec291b07; no voters: 31eca697b43b43199c0142a966b3b61c, 8f9ed5de1a944954a8c1e5361d98719e
I20260811 09:15:09.929530 29229 leader_election.cc:302] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 76ca417693d045288d11ed58ec291b07, 8f9ed5de1a944954a8c1e5361d98719e; no voters: 31eca697b43b43199c0142a966b3b61c
I20260811 09:15:09.929682 30451 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:15:09.929791 30449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:15:09.930265 30449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [term 2 LEADER]: Becoming Leader. State: Replica: 8f9ed5de1a944954a8c1e5361d98719e, State: Running, Role: LEADER
I20260811 09:15:09.930665 30449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } }
I20260811 09:15:09.932245 30466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f9ed5de1a944954a8c1e5361d98719e. Latest consensus state: current_term: 2 leader_uuid: "8f9ed5de1a944954a8c1e5361d98719e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:09.932720 30466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:09.933451 30470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:15:09.937762 30470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:15:09.938563 30470 catalog_manager.cc:1259] Loaded cluster ID: 363259a4cc274759bc3e778fde34f144
I20260811 09:15:09.938660 30470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:15:09.939810 30470 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:15:09.940586 30470 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Loaded TSK: 0
I20260811 09:15:09.941570 30470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:15:09.975401 29256 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:15:09.982354 29256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:15:09.986387 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.985984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:44320)
0811 09:15:09.986135 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.986233 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 09:15:09.986394 29600 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [term 2 FOLLOWER]: Refusing update from remote peer 8f9ed5de1a944954a8c1e5361d98719e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 09:15:09.987066 30449 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 09:15:09.987471 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.987046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:44336)
0811 09:15:09.987222 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.987308 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:15:09.987664 29230 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e -> Peer 31eca697b43b43199c0142a966b3b61c (127.17.16.126:39045): Couldn't send request to peer 31eca697b43b43199c0142a966b3b61c. Status: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 09:15:09.989594 30451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f9ed5de1a944954a8c1e5361d98719e. Latest consensus state: current_term: 2 leader_uuid: "8f9ed5de1a944954a8c1e5361d98719e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:09.989843 30451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:15:09.991394 30449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "8f9ed5de1a944954a8c1e5361d98719e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:09.991653 30449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:09.992385 30473 mvcc.cc:205] Tried to move back new op lower bound from 7317257052097273856 to 7317257051879043072. Current Snapshot: MvccSnapshot[applied={T|T < 7317257052097273856}]
I20260811 09:15:09.992226 30451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "8f9ed5de1a944954a8c1e5361d98719e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "31eca697b43b43199c0142a966b3b61c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 39045 } } peers { permanent_uuid: "8f9ed5de1a944954a8c1e5361d98719e" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 38133 } attrs { promote: false } } peers { permanent_uuid: "76ca417693d045288d11ed58ec291b07" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 44323 } attrs { promote: false } } }
I20260811 09:15:09.992615 30451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ca417693d045288d11ed58ec291b07 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:15:10.013482 30455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.999714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:39355 (local address 127.0.0.1:56138)
0811 09:15:09.999863 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:09.999881 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:09.999990 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:10.007071 (+ 7081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:10.007077 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:10.007106 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:10.007315 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:10.007326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.008798 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.008804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:10.010856 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:10.010872 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.012629 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.012637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:10.012857 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:10.012916 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 09:15:10.013015 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 09:15:10.013176 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 09:15:10.013536 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:09.999843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56138 (local address 127.17.16.66:39355)
0811 09:15:10.006733 (+ 6890us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:10.006739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:10.006760 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:15:10.006837 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:10.006842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:10.006920 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:10.007027 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:10.007495 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.008665 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.011057 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.012397 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.012612 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:10.012725 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:10.013045 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 09:15:10.013190 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6756,"thread_start_us":6596,"threads_started":1}
I20260811 09:15:10.014853 29998 tablet_service.cc:1467] Processing CreateTablet for tablet e54a4cb653a24c4b9279b8ec672ae126 (DEFAULT_TABLE table=test-table [id=94805bd8ffe64b6b899e7e28179e22e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:10.015332 29998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54a4cb653a24c4b9279b8ec672ae126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:10.020061 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.000522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:40501 (local address 127.0.0.1:45012)
0811 09:15:10.001071 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.001087 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:10.001196 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:10.013892 (+ 12696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:10.013897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:10.013924 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:10.014201 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:10.014208 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.015606 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.015611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:10.017659 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:10.017675 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.019263 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.019270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:10.019456 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:10.019490 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:15:10.019557 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 09:15:10.019702 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":592,"threads_started":1}
I20260811 09:15:10.020362 30453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.000099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45197 (local address 127.0.0.1:46986)
0811 09:15:10.000463 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.000480 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:10.000596 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:10.015969 (+ 15373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:10.015973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:10.015998 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:10.016294 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:10.016303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.017327 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.017333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:10.018593 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:10.018604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.019725 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.019732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:10.019879 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:10.019901 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 09:15:10.019976 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 09:15:10.020110 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258}
I20260811 09:15:10.020443 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.000500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46986 (local address 127.17.16.65:45197)
0811 09:15:10.015698 (+ 15198us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:10.015704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:10.015721 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:15:10.015778 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:10.015782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:10.015837 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:10.015929 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:10.016433 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.017211 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.018739 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.019517 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.019626 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:10.019725 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:10.020017 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 09:15:10.020160 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15073,"thread_start_us":15541,"threads_started":1}
I20260811 09:15:10.020994 30482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.000991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.17.16.67:40501)
0811 09:15:10.013400 (+ 12409us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:10.013406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:10.013426 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:15:10.013497 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:10.013503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:10.013583 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:10.014327 (+ 744us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:10.014490 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.015444 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.017868 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.019036 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.020399 (+ 1363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:10.020494 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:10.020546 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 09:15:10.020684 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12274,"thread_start_us":12723,"threads_started":1}
I20260811 09:15:10.021173 29784 tablet_service.cc:1467] Processing CreateTablet for tablet e54a4cb653a24c4b9279b8ec672ae126 (DEFAULT_TABLE table=test-table [id=94805bd8ffe64b6b899e7e28179e22e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:10.021629 29784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54a4cb653a24c4b9279b8ec672ae126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:10.021652 30229 tablet_service.cc:1467] Processing CreateTablet for tablet e54a4cb653a24c4b9279b8ec672ae126 (DEFAULT_TABLE table=test-table [id=94805bd8ffe64b6b899e7e28179e22e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:10.022076 30229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54a4cb653a24c4b9279b8ec672ae126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:10.022825 30484 tablet_bootstrap.cc:492] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445: Bootstrap starting.
I20260811 09:15:10.024677 30484 tablet_bootstrap.cc:654] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:10.026888 30484 tablet_bootstrap.cc:492] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445: No bootstrap required, opened a new log
I20260811 09:15:10.027040 30484 ts_tablet_manager.cc:1397] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 09:15:10.027549 30488 tablet_bootstrap.cc:492] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2: Bootstrap starting.
I20260811 09:15:10.028826 30484 raft_consensus.cc:348] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } } peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } }
I20260811 09:15:10.029145 30484 raft_consensus.cc:374] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:10.029245 30484 raft_consensus.cc:729] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dde75c9995e94b5cbbe56c74a12a9445, State: Initialized, Role: FOLLOWER
I20260811 09:15:10.029407 30489 tablet_bootstrap.cc:492] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2: Bootstrap starting.
I20260811 09:15:10.029511 30484 consensus_queue.cc:260] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } } peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } }
I20260811 09:15:10.029702 30488 tablet_bootstrap.cc:654] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:10.030303 30484 ts_tablet_manager.cc:1428] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:10.031284 30489 tablet_bootstrap.cc:654] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:10.035772 30488 tablet_bootstrap.cc:492] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2: No bootstrap required, opened a new log
I20260811 09:15:10.035923 30488 ts_tablet_manager.cc:1397] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 09:15:10.036499 30489 tablet_bootstrap.cc:492] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2: No bootstrap required, opened a new log
I20260811 09:15:10.036638 30489 ts_tablet_manager.cc:1397] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260811 09:15:10.036721 30488 raft_consensus.cc:348] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } } peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } }
I20260811 09:15:10.036981 30488 raft_consensus.cc:374] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:10.037077 30488 raft_consensus.cc:729] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9170330144184e8385e0338b4b2521c2, State: Initialized, Role: FOLLOWER
I20260811 09:15:10.037349 30488 consensus_queue.cc:260] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } } peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } }
I20260811 09:15:10.037442 30489 raft_consensus.cc:348] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } } peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } }
I20260811 09:15:10.037669 30489 raft_consensus.cc:374] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:10.037775 30489 raft_consensus.cc:729] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be3e7b06847640edaf6a5a221d39a9f2, State: Initialized, Role: FOLLOWER
I20260811 09:15:10.038085 30489 consensus_queue.cc:260] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } } peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } }
I20260811 09:15:10.038411 30496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.030935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.66:57239)
0811 09:15:10.037931 (+ 6996us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.038074 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6781,"thread_start_us":6219,"threads_started":1}
I20260811 09:15:10.040136 30099 heartbeater.cc:503] Master 127.17.16.125:38133 was elected leader, sending a full tablet report...
I20260811 09:15:10.040179 30496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.039805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.66:53385)
0811 09:15:10.039956 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.040039 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 09:15:10.041429 30489 ts_tablet_manager.cc:1428] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260811 09:15:10.044314 30488 ts_tablet_manager.cc:1428] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260811 09:15:10.046207 30501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.045081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.67:46461)
0811 09:15:10.045754 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.045923 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":334,"threads_started":1}
I20260811 09:15:10.047801 30330 heartbeater.cc:503] Master 127.17.16.125:38133 was elected leader, sending a full tablet report...
I20260811 09:15:10.048066 30503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.046910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.67:48927)
0811 09:15:10.047618 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.047816 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":445,"threads_started":1}
I20260811 09:15:10.053771 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.042044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.65:40537)
0811 09:15:10.053336 (+ 11292us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.053475 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11163,"thread_start_us":11038,"threads_started":1}
I20260811 09:15:10.054847 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.054487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.65:60563)
0811 09:15:10.054640 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.054722 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 09:15:10.112864 30491 raft_consensus.cc:482] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:15:10.113145 30491 raft_consensus.cc:504] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } } peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } }
I20260811 09:15:10.113932 30491 leader_election.cc:288] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be3e7b06847640edaf6a5a221d39a9f2 (127.17.16.65:45197), 9170330144184e8385e0338b4b2521c2 (127.17.16.67:40501)
I20260811 09:15:10.114457 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54a4cb653a24c4b9279b8ec672ae126" candidate_uuid: "dde75c9995e94b5cbbe56c74a12a9445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be3e7b06847640edaf6a5a221d39a9f2" is_pre_election: true
I20260811 09:15:10.114506 30261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54a4cb653a24c4b9279b8ec672ae126" candidate_uuid: "dde75c9995e94b5cbbe56c74a12a9445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9170330144184e8385e0338b4b2521c2" is_pre_election: true
I20260811 09:15:10.114681 29813 raft_consensus.cc:2454] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dde75c9995e94b5cbbe56c74a12a9445 in term 0.
I20260811 09:15:10.114778 30261 raft_consensus.cc:2454] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dde75c9995e94b5cbbe56c74a12a9445 in term 0.
I20260811 09:15:10.115162 29934 leader_election.cc:302] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be3e7b06847640edaf6a5a221d39a9f2, dde75c9995e94b5cbbe56c74a12a9445; no voters:
I20260811 09:15:10.115432 30491 raft_consensus.cc:2781] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:15:10.115556 30491 raft_consensus.cc:482] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:15:10.115628 30491 raft_consensus.cc:3037] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:10.117679 30491 raft_consensus.cc:504] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } } peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } }
I20260811 09:15:10.118207 30491 leader_election.cc:288] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [CANDIDATE]: Term 1 election: Requested vote from peers be3e7b06847640edaf6a5a221d39a9f2 (127.17.16.65:45197), 9170330144184e8385e0338b4b2521c2 (127.17.16.67:40501)
I20260811 09:15:10.118548 29813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54a4cb653a24c4b9279b8ec672ae126" candidate_uuid: "dde75c9995e94b5cbbe56c74a12a9445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be3e7b06847640edaf6a5a221d39a9f2"
I20260811 09:15:10.118703 29813 raft_consensus.cc:3037] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:10.118669 30261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54a4cb653a24c4b9279b8ec672ae126" candidate_uuid: "dde75c9995e94b5cbbe56c74a12a9445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9170330144184e8385e0338b4b2521c2"
I20260811 09:15:10.118865 30261 raft_consensus.cc:3037] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:10.121053 29813 raft_consensus.cc:2454] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dde75c9995e94b5cbbe56c74a12a9445 in term 1.
I20260811 09:15:10.121328 30261 raft_consensus.cc:2454] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dde75c9995e94b5cbbe56c74a12a9445 in term 1.
I20260811 09:15:10.121407 29934 leader_election.cc:302] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be3e7b06847640edaf6a5a221d39a9f2, dde75c9995e94b5cbbe56c74a12a9445; no voters:
I20260811 09:15:10.121704 30491 raft_consensus.cc:2781] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:10.122107 30491 raft_consensus.cc:686] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 1 LEADER]: Becoming Leader. State: Replica: dde75c9995e94b5cbbe56c74a12a9445, State: Running, Role: LEADER
I20260811 09:15:10.122400 30491 consensus_queue.cc:237] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } } peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } }
I20260811 09:15:10.123055 30496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.122665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.66:33379)
0811 09:15:10.122814 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.122918 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 09:15:10.124883 29256 catalog_manager.cc:5495] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 reported cstate change: term changed from 0 to 1, leader changed from <none> to dde75c9995e94b5cbbe56c74a12a9445 (127.17.16.66). New cstate: current_term: 1 leader_uuid: "dde75c9995e94b5cbbe56c74a12a9445" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dde75c9995e94b5cbbe56c74a12a9445" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 39355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 } health_report { overall_health: UNKNOWN } } }
I20260811 09:15:10.160660 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.156397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56150 (local address 127.17.16.66:39355)
0811 09:15:10.156570 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:10.156574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:10.156591 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:15:10.156696 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:10.156699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:10.156773 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:10.156843 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:15:10.157165 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.157861 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.159089 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.159505 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.159596 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:10.159916 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:15:10.160020 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:15:10.160081 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:10.160334 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 09:15:10.160413 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:15:10.163991 29813 raft_consensus.cc:1261] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2 [term 1 FOLLOWER]: Refusing update from remote peer dde75c9995e94b5cbbe56c74a12a9445: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:10.164001 30261 raft_consensus.cc:1261] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2 [term 1 FOLLOWER]: Refusing update from remote peer dde75c9995e94b5cbbe56c74a12a9445: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:10.164737 30504 consensus_queue.cc:1035] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9170330144184e8385e0338b4b2521c2" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:10.165138 30491 consensus_queue.cc:1035] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be3e7b06847640edaf6a5a221d39a9f2" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:10.166879 29885 heartbeater.cc:503] Master 127.17.16.125:38133 was elected leader, sending a full tablet report...
I20260811 09:15:10.167491 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.167042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.65:39683)
0811 09:15:10.167186 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.167293 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 09:15:10.168133 30503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.167703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.67:37201)
0811 09:15:10.167841 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.167937 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 09:15:10.173769 30507 mvcc.cc:205] Tried to move back new op lower bound from 7317257052825182208 to 7317257052663435264. Current Snapshot: MvccSnapshot[applied={T|T < 7317257052825182208}]
I20260811 09:15:10.177161 30496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.176601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.66:51655)
0811 09:15:10.176829 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.176969 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 09:15:10.177258 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.176407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.65:47839)
0811 09:15:10.176980 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.177081 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413}
I20260811 09:15:10.178284 30496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.177789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.66:37465)
0811 09:15:10.178005 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.178090 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 09:15:10.181505 30503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.179712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.17.16.67:37411)
0811 09:15:10.181187 (+ 1475us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.181304 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1362}
I20260811 09:15:10.188654 29253 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:10.188917 29253 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:10.194185 29253 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f9ed5de1a944954a8c1e5361d98719e: Sending DeleteTablet for 3 replicas of tablet e54a4cb653a24c4b9279b8ec672ae126
I20260811 09:15:10.195170 29998 tablet_service.cc:1514] Processing DeleteTablet for tablet e54a4cb653a24c4b9279b8ec672ae126 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:10 UTC) from {username='oryx', principal='oryx/127.17.16.125@KRBTEST.COM'} at 127.0.0.1:56138
I20260811 09:15:10.195573 29784 tablet_service.cc:1514] Processing DeleteTablet for tablet e54a4cb653a24c4b9279b8ec672ae126 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:10 UTC) from {username='oryx', principal='oryx/127.17.16.125@KRBTEST.COM'} at 127.0.0.1:46986
I20260811 09:15:10.195816 30519 tablet_replica.cc:330] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445: stopping tablet replica
I20260811 09:15:10.195834 30229 tablet_service.cc:1514] Processing DeleteTablet for tablet e54a4cb653a24c4b9279b8ec672ae126 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:10 UTC) from {username='oryx', principal='oryx/127.17.16.125@KRBTEST.COM'} at 127.0.0.1:45012
I20260811 09:15:10.195987 30519 raft_consensus.cc:2229] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:15:10.196413 30519 raft_consensus.cc:2258] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:10.196619 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 29531
I20260811 09:15:10.197458 30519 ts_tablet_manager.cc:1905] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:10.204413 30519 ts_tablet_manager.cc:1918] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:10.204564 30519 log.cc:1196] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/e54a4cb653a24c4b9279b8ec672ae126
I20260811 09:15:10.205034 30519 ts_tablet_manager.cc:1939] T e54a4cb653a24c4b9279b8ec672ae126 P dde75c9995e94b5cbbe56c74a12a9445: Deleting consensus metadata
I20260811 09:15:10.206156 29229 catalog_manager.cc:4841] TS dde75c9995e94b5cbbe56c74a12a9445 (127.17.16.66:39355): tablet e54a4cb653a24c4b9279b8ec672ae126 (table test-table [id=94805bd8ffe64b6b899e7e28179e22e9]) successfully deleted
I20260811 09:15:10.206502 30523 tablet_replica.cc:330] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2: stopping tablet replica
I20260811 09:15:10.206656 30523 raft_consensus.cc:2229] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:10.206876 30523 raft_consensus.cc:2258] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:10.207753 30523 ts_tablet_manager.cc:1905] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:10.213019 30523 ts_tablet_manager.cc:1918] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:10.213147 30523 log.cc:1196] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/e54a4cb653a24c4b9279b8ec672ae126
I20260811 09:15:10.213474 30523 ts_tablet_manager.cc:1939] T e54a4cb653a24c4b9279b8ec672ae126 P 9170330144184e8385e0338b4b2521c2: Deleting consensus metadata
I20260811 09:15:10.214511 29230 catalog_manager.cc:4841] TS 9170330144184e8385e0338b4b2521c2 (127.17.16.67:40501): tablet e54a4cb653a24c4b9279b8ec672ae126 (table test-table [id=94805bd8ffe64b6b899e7e28179e22e9]) successfully deleted
I20260811 09:15:10.214695 30520 tablet_replica.cc:330] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2: stopping tablet replica
I20260811 09:15:10.214874 30520 raft_consensus.cc:2229] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:10.215147 30520 raft_consensus.cc:2258] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:10.216056 30520 ts_tablet_manager.cc:1905] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:10.221678 30520 ts_tablet_manager.cc:1918] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:10.221788 30520 log.cc:1196] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/e54a4cb653a24c4b9279b8ec672ae126
I20260811 09:15:10.222132 30520 ts_tablet_manager.cc:1939] T e54a4cb653a24c4b9279b8ec672ae126 P be3e7b06847640edaf6a5a221d39a9f2: Deleting consensus metadata
I20260811 09:15:10.223096 29233 catalog_manager.cc:4841] TS be3e7b06847640edaf6a5a221d39a9f2 (127.17.16.65:45197): tablet e54a4cb653a24c4b9279b8ec672ae126 (table test-table [id=94805bd8ffe64b6b899e7e28179e22e9]) successfully deleted
I20260811 09:15:10.224527 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.199324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.17.16.65:45197)
0811 09:15:10.199533 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:10.199539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:10.208632 (+ 9093us) server_negotiation.cc:407] Connection header received
0811 09:15:10.208831 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:10.208836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:10.208957 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:10.209049 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:10.211071 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.212269 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.214556 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.215212 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.215344 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:10.218675 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:10.218689 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:10.218705 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:10.218739 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:10.221773 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:10.222526 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:10.222534 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:10.222541 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:10.222699 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:10.223066 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:10.223072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:10.223075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:10.223446 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:10.223730 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:10.224009 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 09:15:10.224232 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 09:15:10.225193 29872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:10.317166 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 29531
I20260811 09:15:10.340235 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 29904
I20260811 09:15:10.353034 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.341019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56164 (local address 127.17.16.66:39355)
0811 09:15:10.341202 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:10.341206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:10.341228 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:15:10.341389 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:10.341393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:10.341497 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:10.341585 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:10.342863 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.343564 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.344704 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.345146 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.345239 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:10.347774 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:10.347789 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:10.347807 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:10.347838 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:10.350726 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:10.351324 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:10.351331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:10.351333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:10.351458 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:10.351746 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:10.351749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:10.351751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:10.352098 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:10.352338 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:10.352530 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 09:15:10.352698 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:15:10.353665 30089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:10.403949 30453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.403459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:44340)
0811 09:15:10.403643 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.403756 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 09:15:10.405050 30453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.404606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:39045 (local address 127.0.0.1:44344)
0811 09:15:10.404757 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:10.404840 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 09:15:10.473667 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 29904
I20260811 09:15:10.503638 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 30115
I20260811 09:15:10.516323 30482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:10.504470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.17.16.67:40501)
0811 09:15:10.504667 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:10.504672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:10.504690 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:15:10.504807 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:10.504811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:10.504875 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:10.504984 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:10.506371 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.507053 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.508244 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:10.508653 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:10.508739 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:10.511238 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:10.511253 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:10.511270 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:10.511305 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:10.513868 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:10.514573 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:10.514579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:10.514582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:10.514696 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:10.514964 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:10.514967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:10.514969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:10.515315 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:10.515587 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:10.515764 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 09:15:10.515980 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:15:10.517023 30316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:10.607476 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 30115
I20260811 09:15:10.630546 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 29022
I20260811 09:15:10.631633 29307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:10.753316 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 29022
W20260811 09:15:10.781697 29551 connection.cc:535] server connection from 127.0.0.1:41976 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 09:15:10.782258 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 29193
I20260811 09:15:10.783260 29628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:10.899027 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 29193
2026-08-11T09:15:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9703 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:15:11 dist-test-slave-ncp9 krb5kdc[30539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:15:11 dist-test-slave-ncp9 krb5kdc[30539](info): set up 2 sockets
Aug 11 09:15:11 dist-test-slave-ncp9 krb5kdc[30539](info): commencing operation
krb5kdc: starting...
W20260811 09:15:12.144178 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:15:12 dist-test-slave-ncp9 krb5kdc[30539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:15:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:15:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.126.keytab.
Entry for principal oryx/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.126.keytab.
I20260811 09:15:12.390758 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46579
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.126.keytab
--principal=oryx/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:46579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:12.712047 30558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:12.712347 30558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:12.712486 30558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:12.719545 30558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:15:12.719664 30558 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 09:15:12.719730 30558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:12.719813 30558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:12.719882 30558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:15:12.719976 30558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:15:12.729868 30558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:46579
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.126.keytab
--principal=oryx/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:12.731137 30558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:12.732455 30558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:12.743841 30565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:12.743942 30573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:12.743912 30564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:12.744699 30558 server_base.cc:1034] running on GCE node
I20260811 09:15:12.745510 30558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:12.746850 30558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:12.748068 30558 hybrid_clock.cc:648] HybridClock initialized: now 1786439712748008 us; error 31 us; skew 500 ppm
I20260811 09:15:12.748365 30558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:12 dist-test-slave-ncp9 krb5kdc[30539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439712, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:12.751263 30558 init.cc:377] Logged in from keytab as oryx/127.17.16.126@KRBTEST.COM (short username oryx)
I20260811 09:15:12.752636 30558 webserver.cc:460] Webserver started at http://127.17.16.126:33597/ using document root <none> and password file <none>
I20260811 09:15:12.753322 30558 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:12.753420 30558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:12.753732 30558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:12.756536 30558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "6337b7c373864632bbaecd34168e1ce4"
format_stamp: "Formatted at 2026-08-11 09:15:12 on dist-test-slave-ncp9"
server_key: "f87adcf20ac6fe6d3e0883f60cf33104"
server_key_iv: "12d079d72dcd186954a6bf85fa5589cf"
server_key_version: "encryptionkey@0"
I20260811 09:15:12.757305 30558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "6337b7c373864632bbaecd34168e1ce4"
format_stamp: "Formatted at 2026-08-11 09:15:12 on dist-test-slave-ncp9"
server_key: "f87adcf20ac6fe6d3e0883f60cf33104"
server_key_iv: "12d079d72dcd186954a6bf85fa5589cf"
server_key_version: "encryptionkey@0"
I20260811 09:15:12.761996 30558 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:15:12.765213 30588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:12.766382 30558 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 09:15:12.766551 30558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "6337b7c373864632bbaecd34168e1ce4"
format_stamp: "Formatted at 2026-08-11 09:15:12 on dist-test-slave-ncp9"
server_key: "f87adcf20ac6fe6d3e0883f60cf33104"
server_key_iv: "12d079d72dcd186954a6bf85fa5589cf"
server_key_version: "encryptionkey@0"
I20260811 09:15:12.766744 30558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:12.786265 30558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:12.790052 30558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:12.790297 30558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:12.809789 30558 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:46579
I20260811 09:15:12.809808 30688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:46579 every 8 connection(s)
I20260811 09:15:12.811036 30558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:15:12.811349 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 30558
I20260811 09:15:12.811548 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:15:12.812341 17473 external_mini_cluster.cc:1428] Setting key d250f6d820ecd4471422a9dc26d91b2e
I20260811 09:15:12.815320 30689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:15:12 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:12.825876 30689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4: Bootstrap starting.
I20260811 09:15:12.829332 30689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:12.830367 30689 log.cc:824] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:12.833369 30689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4: No bootstrap required, opened a new log
I20260811 09:15:12.834088 30693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:12.814357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.17.16.126:46579)
0811 09:15:12.815177 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:12.815189 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:12.815233 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:15:12.816919 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:12.816949 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:12.817330 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:12.817689 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:12.819180 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:12.820537 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:12.821816 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:12.822509 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:12.822680 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:12.826608 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:12.826618 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:12.826633 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:12.826676 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:12.829790 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:12.830468 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:12.830475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:12.830482 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:12.830677 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:12.831041 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:12.831046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:12.831049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:12.831603 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:12.831890 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:12.832351 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 09:15:12.832813 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":369,"threads_started":1}
I20260811 09:15:12.841164 30689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337b7c373864632bbaecd34168e1ce4" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46579 } }
I20260811 09:15:12.841495 30689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:12.841572 30689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6337b7c373864632bbaecd34168e1ce4, State: Initialized, Role: FOLLOWER
I20260811 09:15:12.842078 30689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337b7c373864632bbaecd34168e1ce4" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46579 } }
I20260811 09:15:12.842299 30689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:15:12.842396 30689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:15:12.842525 30689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:12.844514 30689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337b7c373864632bbaecd34168e1ce4" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46579 } }
I20260811 09:15:12.845005 30689 leader_election.cc:302] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6337b7c373864632bbaecd34168e1ce4; no voters:
I20260811 09:15:12.845521 30689 leader_election.cc:288] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:15:12.845657 30700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:12.846272 30700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [term 1 LEADER]: Becoming Leader. State: Replica: 6337b7c373864632bbaecd34168e1ce4, State: Running, Role: LEADER
I20260811 09:15:12.846777 30700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337b7c373864632bbaecd34168e1ce4" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46579 } }
I20260811 09:15:12.847394 30689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:15:12.849488 30701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6337b7c373864632bbaecd34168e1ce4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337b7c373864632bbaecd34168e1ce4" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46579 } } }
I20260811 09:15:12.849510 30702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6337b7c373864632bbaecd34168e1ce4. Latest consensus state: current_term: 1 leader_uuid: "6337b7c373864632bbaecd34168e1ce4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6337b7c373864632bbaecd34168e1ce4" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46579 } } }
I20260811 09:15:12.849918 30701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:12.850051 30702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:12.851531 30707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:15:12.856598 30707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:15:12.864518 30707 catalog_manager.cc:1347] Generated new cluster ID: a17a1113643d47c0ba8ed90838258f8a
I20260811 09:15:12.864634 30707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:15:12.893810 30707 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:15:12.895159 30707 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:15:12.903595 30707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4: Generated new TSK 0
I20260811 09:15:12.904392 30707 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab.
Entry for principal oryx/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab.
I20260811 09:15:12.969544 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46579
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab
--principal=oryx/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:13.315033 30734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:13.315311 30734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:13.315408 30734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:13.323092 30734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:13.323221 30734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:13.323312 30734 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:15:13.323405 30734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:15:13.333804 30734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab
--principal=oryx/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.16.126:46579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:13.335189 30734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:13.336638 30734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:13.350785 30741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:13.350816 30747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:13.350731 30750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:13.351845 30734 server_base.cc:1034] running on GCE node
I20260811 09:15:13.352459 30734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:13.353426 30734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:13.354645 30734 hybrid_clock.cc:648] HybridClock initialized: now 1786439713354580 us; error 33 us; skew 500 ppm
I20260811 09:15:13.354928 30734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:13 dist-test-slave-ncp9 krb5kdc[30539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439713, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:13.358021 30734 init.cc:377] Logged in from keytab as oryx/127.17.16.65@KRBTEST.COM (short username oryx)
I20260811 09:15:13.359424 30734 webserver.cc:460] Webserver started at http://127.17.16.65:44217/ using document root <none> and password file <none>
I20260811 09:15:13.360086 30734 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:13.360181 30734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:13.360455 30734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:13.363438 30734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "a17f8ccef9f644f4a3d8a570baf6561d"
format_stamp: "Formatted at 2026-08-11 09:15:13 on dist-test-slave-ncp9"
server_key: "cf399fe9ccb9ff5b32fa1bbca5df8ff6"
server_key_iv: "3812b8aaf30341bf693e53b6860d7f69"
server_key_version: "encryptionkey@0"
I20260811 09:15:13.364171 30734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "a17f8ccef9f644f4a3d8a570baf6561d"
format_stamp: "Formatted at 2026-08-11 09:15:13 on dist-test-slave-ncp9"
server_key: "cf399fe9ccb9ff5b32fa1bbca5df8ff6"
server_key_iv: "3812b8aaf30341bf693e53b6860d7f69"
server_key_version: "encryptionkey@0"
I20260811 09:15:13.369234 30734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 09:15:13.372674 30763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:13.374013 30734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:13.374179 30734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "a17f8ccef9f644f4a3d8a570baf6561d"
format_stamp: "Formatted at 2026-08-11 09:15:13 on dist-test-slave-ncp9"
server_key: "cf399fe9ccb9ff5b32fa1bbca5df8ff6"
server_key_iv: "3812b8aaf30341bf693e53b6860d7f69"
server_key_version: "encryptionkey@0"
I20260811 09:15:13.374392 30734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:13.389434 30734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:13.393374 30734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:13.393626 30734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:13.396255 30734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:13.396341 30734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:13.396529 30734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:13.396606 30734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:15:13 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439713, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:13.420521 30779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:13.397610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46579 (local address 127.17.16.65:60809)
0811 09:15:13.398473 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:13.398523 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:13.400449 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:13.401230 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:13.401238 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:13.401965 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:13.402855 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:13.402882 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.404228 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.404232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:13.406351 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:13.406365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.407512 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.407525 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:13.407800 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:13.408633 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:13.408671 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:13.411750 (+ 3079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:13.415313 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:13.415325 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:13.415349 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:13.416051 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:13.416456 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:13.416472 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:13.416478 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:13.416779 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:13.417562 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:13.417571 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:13.418111 (+ 540us) client_negotiation.cc:769] Sending connection context
0811 09:15:13.418467 (+ 356us) client_negotiation.cc:240] Negotiation successful
0811 09:15:13.419006 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":346,"threads_started":1}
I20260811 09:15:13.420950 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:13.396983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:60809 (local address 127.17.16.126:46579)
0811 09:15:13.397577 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:13.397583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:13.398658 (+ 1075us) server_negotiation.cc:407] Connection header received
0811 09:15:13.400741 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:13.400747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:13.400840 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:13.400983 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:13.403038 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.404044 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.406530 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.407195 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.407349 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:13.411989 (+ 4640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:13.411995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:13.411999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:13.412033 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:13.415117 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:13.416186 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:13.416192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:13.416196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:13.416333 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:13.416971 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:13.416977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:13.416981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:13.417308 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:13.417479 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:13.420268 (+ 2789us) server_negotiation.cc:299] Negotiation successful
0811 09:15:13.420540 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":330,"threads_started":1}
I20260811 09:15:13.441617 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:13.427462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:46981 (local address 127.17.16.126:46579)
0811 09:15:13.427658 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:13.427664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:13.428179 (+ 515us) server_negotiation.cc:407] Connection header received
0811 09:15:13.428315 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:13.428321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:13.428406 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:13.428518 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:13.429689 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.430906 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.432222 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.432936 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.433069 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:13.435115 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:13.435124 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:13.435129 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:13.435172 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:13.438430 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:13.439365 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:13.439370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:13.439374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:13.439534 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:13.439930 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:13.439935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:13.439939 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:13.440265 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:13.440405 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:13.441003 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 09:15:13.441230 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:15:13.441701 30912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:13.427453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46579 (local address 127.17.16.65:46981)
0811 09:15:13.428090 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:13.428107 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:13.428220 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:13.428533 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:13.428536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:13.429003 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:13.429425 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:13.429442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.431048 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.431052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:13.432099 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:13.432113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.433103 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.433108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:13.433233 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:13.433928 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:13.433952 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:13.434903 (+ 951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:13.438623 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:13.438628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:13.438636 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:13.439238 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:13.439662 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:13.439666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:13.439668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:13.439810 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:13.440468 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:13.440472 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:13.440705 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 09:15:13.440885 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 09:15:13.441253 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":321,"threads_started":1}
I20260811 09:15:13.451109 30734 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:41163
I20260811 09:15:13.451130 30951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:41163 every 8 connection(s)
I20260811 09:15:13.452332 30734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:15:13.454459 30953 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46579
I20260811 09:15:13.454680 30953 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:13.455356 30953 heartbeater.cc:511] Master 127.17.16.126:46579 requested a full tablet report, sending...
I20260811 09:15:13.457351 30623 ts_manager.cc:194] Registered new tserver with Master: a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65:41163)
I20260811 09:15:13.458871 30623 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:60809
I20260811 09:15:13.461140 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 30734
I20260811 09:15:13.461357 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:15:13.462064 17473 external_mini_cluster.cc:1428] Setting key e513b5c3e693d57118d031968ff5a5dc
WARNING: no policy specified for oryx/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab.
Entry for principal oryx/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab.
I20260811 09:15:13.518080 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46579
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab
--principal=oryx/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:13.847724 30960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:13.848012 30960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:13.848171 30960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:13.855603 30960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:13.855762 30960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:13.855901 30960 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:15:13.856029 30960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:15:13.866420 30960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab
--principal=oryx/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.16.126:46579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:13.867718 30960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:13.869094 30960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:13.882409 30967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:13.882480 30966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:13.882447 30970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:13.883047 30960 server_base.cc:1034] running on GCE node
I20260811 09:15:13.883637 30960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:13.884496 30960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:13.885696 30960 hybrid_clock.cc:648] HybridClock initialized: now 1786439713885638 us; error 33 us; skew 500 ppm
I20260811 09:15:13.885964 30960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:13 dist-test-slave-ncp9 krb5kdc[30539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439713, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:13.888878 30960 init.cc:377] Logged in from keytab as oryx/127.17.16.66@KRBTEST.COM (short username oryx)
I20260811 09:15:13.890265 30960 webserver.cc:460] Webserver started at http://127.17.16.66:42843/ using document root <none> and password file <none>
I20260811 09:15:13.890908 30960 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:13.891008 30960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:13.891326 30960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:13.894282 30960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "2f560081c29641c88dad14da789187a5"
format_stamp: "Formatted at 2026-08-11 09:15:13 on dist-test-slave-ncp9"
server_key: "963908dc039ee4c61c380a6ce0329173"
server_key_iv: "6c6309322b5d7919574548bfea2494d2"
server_key_version: "encryptionkey@0"
I20260811 09:15:13.895195 30960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "2f560081c29641c88dad14da789187a5"
format_stamp: "Formatted at 2026-08-11 09:15:13 on dist-test-slave-ncp9"
server_key: "963908dc039ee4c61c380a6ce0329173"
server_key_iv: "6c6309322b5d7919574548bfea2494d2"
server_key_version: "encryptionkey@0"
I20260811 09:15:13.900552 30960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:15:13.904119 30981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:13.905396 30960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:13.905541 30960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "2f560081c29641c88dad14da789187a5"
format_stamp: "Formatted at 2026-08-11 09:15:13 on dist-test-slave-ncp9"
server_key: "963908dc039ee4c61c380a6ce0329173"
server_key_iv: "6c6309322b5d7919574548bfea2494d2"
server_key_version: "encryptionkey@0"
I20260811 09:15:13.905737 30960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:13.924727 30960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:13.928277 30960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:13.928515 30960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:13.931109 30960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:13.931201 30960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:13.931341 30960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:13.931401 30960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:15:13 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439713, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:13.954741 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:13.931850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:47335 (local address 127.17.16.126:46579)
0811 09:15:13.932036 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:13.932043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:13.933802 (+ 1759us) server_negotiation.cc:407] Connection header received
0811 09:15:13.936119 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:13.936128 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:13.936233 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:13.936357 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:13.938881 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.940022 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.942611 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.943302 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.943434 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:13.947631 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:13.947647 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:13.947652 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:13.947697 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:13.951069 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:13.952065 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:13.952070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:13.952074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:13.952214 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:13.952632 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:13.952638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:13.952642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:13.952975 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:13.953124 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:13.954077 (+ 953us) server_negotiation.cc:299] Negotiation successful
0811 09:15:13.954328 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:15:13.955545 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:13.932418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46579 (local address 127.17.16.66:47335)
0811 09:15:13.933592 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:13.933648 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:13.935726 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:13.936674 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:13.936684 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:13.937547 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:13.938577 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:13.938606 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.940244 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.940250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:13.942435 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:13.942456 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.943635 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.943649 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:13.943875 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:13.944571 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:13.944595 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:13.947379 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:13.951249 (+ 3870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:13.951258 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:13.951278 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:13.951929 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:13.952321 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:13.952327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:13.952331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:13.952516 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:13.953103 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:13.953111 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:13.953600 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 09:15:13.953958 (+ 358us) client_negotiation.cc:240] Negotiation successful
0811 09:15:13.954409 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":618,"threads_started":1}
I20260811 09:15:13.974609 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:13.961035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:42623 (local address 127.17.16.126:46579)
0811 09:15:13.961199 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:13.961204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:13.961845 (+ 641us) server_negotiation.cc:407] Connection header received
0811 09:15:13.961996 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:13.962000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:13.962061 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:13.962138 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:13.963072 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.963768 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.965058 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.965704 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.965833 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:13.968184 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:13.968190 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:13.968193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:13.968224 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:13.971704 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:13.972539 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:13.972545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:13.972549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:13.972735 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:13.973209 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:13.973214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:13.973217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:13.973539 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:13.973656 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:13.974190 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 09:15:13.974363 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:15:13.974890 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:13.961131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46579 (local address 127.17.16.66:42623)
0811 09:15:13.961742 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:13.961761 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:13.961886 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:13.962174 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:13.962179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:13.962644 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:13.962921 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:13.962929 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.963899 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.963904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:13.964879 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:13.964933 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:13.965929 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:13.965936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:13.966187 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:13.966885 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:13.966909 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:13.967980 (+ 1071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:13.971828 (+ 3848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:13.971832 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:13.971835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:13.972430 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:13.972871 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:13.972876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:13.972881 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:13.973076 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:13.973659 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:13.973664 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:13.973882 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 09:15:13.974031 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 09:15:13.974398 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":353,"threads_started":1}
I20260811 09:15:13.989297 30960 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:33947
I20260811 09:15:13.989318 31181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:33947 every 8 connection(s)
I20260811 09:15:13.990465 30960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:15:13.992091 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 30960
I20260811 09:15:13.992297 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:15:13.992385 31183 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46579
I20260811 09:15:13.992565 31183 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:13.992750 17473 external_mini_cluster.cc:1428] Setting key bc1322f629b4ceec36122046ca18bb59
I20260811 09:15:13.993180 31183 heartbeater.cc:511] Master 127.17.16.126:46579 requested a full tablet report, sending...
I20260811 09:15:13.994452 30623 ts_manager.cc:194] Registered new tserver with Master: 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947)
I20260811 09:15:13.995533 30623 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:47335
WARNING: no policy specified for oryx/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab.
Entry for principal oryx/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab.
I20260811 09:15:14.047911 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46579
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab
--principal=oryx/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:14.376657 31191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:14.376991 31191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:14.377122 31191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:14.384407 31191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:14.384513 31191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:14.384625 31191 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:15:14.384714 31191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:15:14.394434 31191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab
--principal=oryx/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.16.126:46579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:14.395699 31191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:14.397018 31191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:14.409428 31200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:14.409528 31199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:14.409528 31202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:14.410421 31191 server_base.cc:1034] running on GCE node
I20260811 09:15:14.411012 31191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:14.411870 31191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:14.413069 31191 hybrid_clock.cc:648] HybridClock initialized: now 1786439714412998 us; error 44 us; skew 500 ppm
I20260811 09:15:14.413352 31191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:14 dist-test-slave-ncp9 krb5kdc[30539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439714, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:14.416534 31191 init.cc:377] Logged in from keytab as oryx/127.17.16.67@KRBTEST.COM (short username oryx)
I20260811 09:15:14.418123 31191 webserver.cc:460] Webserver started at http://127.17.16.67:45087/ using document root <none> and password file <none>
I20260811 09:15:14.418828 31191 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:14.418936 31191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:14.419257 31191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:14.422390 31191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "2e3c5642bc4147eea8533ff8b62add6b"
format_stamp: "Formatted at 2026-08-11 09:15:14 on dist-test-slave-ncp9"
server_key: "75dc506f408bb985f78c45dc992e2a3f"
server_key_iv: "9a0884d650702be46d3e4ca9827fd048"
server_key_version: "encryptionkey@0"
I20260811 09:15:14.423206 31191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "2e3c5642bc4147eea8533ff8b62add6b"
format_stamp: "Formatted at 2026-08-11 09:15:14 on dist-test-slave-ncp9"
server_key: "75dc506f408bb985f78c45dc992e2a3f"
server_key_iv: "9a0884d650702be46d3e4ca9827fd048"
server_key_version: "encryptionkey@0"
I20260811 09:15:14.428188 31191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:15:14.431545 31216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:14.432798 31191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:15:14.432967 31191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "2e3c5642bc4147eea8533ff8b62add6b"
format_stamp: "Formatted at 2026-08-11 09:15:14 on dist-test-slave-ncp9"
server_key: "75dc506f408bb985f78c45dc992e2a3f"
server_key_iv: "9a0884d650702be46d3e4ca9827fd048"
server_key_version: "encryptionkey@0"
I20260811 09:15:14.433164 31191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:14.454882 31191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:14.459115 31191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:14.459390 31191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:14.461730 30953 heartbeater.cc:503] Master 127.17.16.126:46579 was elected leader, sending a full tablet report...
I20260811 09:15:14.462630 31191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:14.462816 31191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 09:15:14.463049 31191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:14.463183 31191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:15:14 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439714, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:14.485688 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:14.463013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:48863 (local address 127.17.16.126:46579)
0811 09:15:14.463230 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:14.463236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:14.464689 (+ 1453us) server_negotiation.cc:407] Connection header received
0811 09:15:14.467377 (+ 2688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:14.467382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:14.467445 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:14.467537 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:14.469888 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:14.470878 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:14.473719 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:14.474241 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:14.474369 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:14.479514 (+ 5145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:14.479520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:14.479523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:14.479561 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:14.482648 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:14.483343 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:14.483349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:14.483353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:14.483489 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:14.483895 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:14.483900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:14.483904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:14.484183 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:14.484318 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:14.485097 (+ 779us) server_negotiation.cc:299] Negotiation successful
0811 09:15:14.485329 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 09:15:14.486706 31234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:14.463547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46579 (local address 127.17.16.67:48863)
0811 09:15:14.464507 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:14.464563 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:14.467058 (+ 2495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:14.467817 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:14.467827 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:14.468680 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:14.469664 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:14.469698 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:14.471049 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:14.471056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:14.473552 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:14.473573 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:14.474674 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:14.474682 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:14.474994 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:14.475978 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:14.476005 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:14.479253 (+ 3248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:14.482796 (+ 3543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:14.482805 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:14.482820 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:14.483217 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:14.483595 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:14.483600 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:14.483604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:14.483780 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:14.484309 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:14.484317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:14.484654 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 09:15:14.484884 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 09:15:14.485464 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":373,"threads_started":1}
I20260811 09:15:14.508108 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:14.493030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:53883 (local address 127.17.16.126:46579)
0811 09:15:14.493183 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:14.493187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:14.494075 (+ 888us) server_negotiation.cc:407] Connection header received
0811 09:15:14.494301 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:14.494306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:14.494383 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:14.494495 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:14.495762 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:14.496522 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:14.498639 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:14.499070 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:14.499172 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:14.501635 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:14.501642 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:14.501646 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:14.501682 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:14.504437 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:14.505471 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:14.505479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:14.505483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:14.505652 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:14.506192 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:14.506198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:14.506202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:14.506582 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:14.506734 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:14.507463 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 09:15:14.507692 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 09:15:14.508209 31328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:14.493048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46579 (local address 127.17.16.67:53883)
0811 09:15:14.493952 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:14.493975 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:14.494154 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:14.494524 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:14.494529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:14.495192 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:14.495595 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:14.495608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:14.496661 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:14.496667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:14.498472 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:14.498494 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:14.499270 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:14.499277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:14.499485 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:14.500186 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:14.500209 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:14.501401 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:14.504623 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:14.504629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:14.504634 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:14.505280 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:14.505787 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:14.505794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:14.505798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:14.506010 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:14.506823 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:14.506827 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:14.507081 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 09:15:14.507320 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 09:15:14.507712 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":405,"threads_started":1}
I20260811 09:15:14.519127 31191 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:35683
I20260811 09:15:14.519150 31409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:35683 every 8 connection(s)
I20260811 09:15:14.520336 31191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:15:14.522343 31411 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46579
I20260811 09:15:14.522550 31411 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:14.522639 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 31191
I20260811 09:15:14.522845 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:15:14.523255 31411 heartbeater.cc:511] Master 127.17.16.126:46579 requested a full tablet report, sending...
I20260811 09:15:14.523275 17473 external_mini_cluster.cc:1428] Setting key 5ff67a456aa193afdda66ff6b3040015
I20260811 09:15:14.524668 30617 ts_manager.cc:194] Registered new tserver with Master: 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683)
I20260811 09:15:14.525671 30617 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:48863
I20260811 09:15:14.526324 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:15:14 dist-test-slave-ncp9 krb5kdc[30539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439714, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:15:14 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:14 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:14.998000 31183 heartbeater.cc:503] Master 127.17.16.126:46579 was elected leader, sending a full tablet report...
I20260811 09:15:15.528070 31411 heartbeater.cc:503] Master 127.17.16.126:46579 was elected leader, sending a full tablet report...
W20260811 09:15:15.599426 31420 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.019s user 0.009s sys 0.009s
W20260811 09:15:15.600683 30778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:14.573246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48474 (local address 127.17.16.126:46579)
0811 09:15:14.573465 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:14.573508 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:14.573970 (+ 462us) server_negotiation.cc:407] Connection header received
0811 09:15:14.574198 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:14.574205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:14.574282 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:14.574371 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:14.576137 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:14.577033 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:14.578454 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:14.578951 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:14.579069 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.600375 (+1021306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48474: BlockingRecv error: recv got EOF from 127.0.0.1:48474 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 09:15:15 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439714, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:15.619912 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.604120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48484 (local address 127.17.16.126:46579)
0811 09:15:15.604336 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:15.604343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:15.604591 (+ 248us) server_negotiation.cc:407] Connection header received
0811 09:15:15.604777 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:15.604782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:15.604869 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:15.604987 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:15.606188 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.606920 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.608153 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.608580 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.608704 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.615211 (+ 6507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:15.615218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:15.615221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:15.615260 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:15.617828 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:15.618413 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:15.618416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:15.618418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:15.618540 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:15.618820 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:15.618823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:15.618825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:15.619100 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:15.619320 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:15.619490 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 09:15:15.619688 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:15:15.624013 30617 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:15:15.630690 30617 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:15:15.667323 31449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.642731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.17.16.67:35683)
0811 09:15:15.659096 (+ 16365us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:15.659105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:15.659147 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:15:15.659267 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:15.659274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:15.659539 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:15.659731 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:15.660677 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.662188 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.664753 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.666244 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.666359 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.666562 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:15.666858 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 09:15:15.667022 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16191,"thread_start_us":17153,"threads_started":1}
I20260811 09:15:15.667390 31454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.642748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.17.16.65:41163)
0811 09:15:15.653466 (+ 10718us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:15.653474 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:15.659109 (+ 5635us) server_negotiation.cc:407] Connection header received
0811 09:15:15.660160 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:15.660169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:15.660390 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:15.660546 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:15.661453 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.662694 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.665101 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.666043 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.666179 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.666394 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:15.666907 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 09:15:15.667093 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10532,"thread_start_us":9525,"threads_started":1}
I20260811 09:15:15.668273 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.642167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:35683 (local address 127.0.0.1:40740)
0811 09:15:15.658624 (+ 16457us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:15.658681 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:15.658942 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:15.659894 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:15.659901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:15.659932 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:15.660412 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:15.660427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.662335 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.662340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:15.664604 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:15.664620 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.666434 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.666440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:15.666625 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.666680 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 09:15:15.666798 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 09:15:15.666977 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16111,"mutex_wait_us":4,"thread_start_us":18042,"threads_started":1}
I20260811 09:15:15.668329 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.642364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:41163 (local address 127.0.0.1:58668)
0811 09:15:15.658948 (+ 16584us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:15.658969 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:15.659910 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:15.660804 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:15.660808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:15.660829 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:15.661277 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:15.661291 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.662837 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.662842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:15.664951 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:15.664965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.666539 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.666546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:15.666703 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.666737 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:15:15.667693 (+ 956us) client_negotiation.cc:240] Negotiation successful
0811 09:15:15.667826 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16436,"mutex_wait_us":4,"thread_start_us":16795,"threads_started":1}
I20260811 09:15:15.669028 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.642654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:33947 (local address 127.0.0.1:53794)
0811 09:15:15.660513 (+ 17859us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:15.660530 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:15.660731 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:15.661573 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:15.661577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:15.661599 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:15.661918 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:15.661929 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.663799 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.663804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:15.666006 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:15.666022 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.668291 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.668299 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:15.668461 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.668496 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:15:15.668583 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 09:15:15.668705 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17735,"thread_start_us":16504,"threads_started":1}
I20260811 09:15:15.669436 31455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.643310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.17.16.66:33947)
0811 09:15:15.653130 (+ 9820us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:15.653141 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:15.660628 (+ 7487us) server_negotiation.cc:407] Connection header received
0811 09:15:15.660960 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:15.660968 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:15.661190 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:15.661341 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:15.662112 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.663375 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.666175 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.667674 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.667813 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.668002 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:15.669004 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0811 09:15:15.669166 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9616,"thread_start_us":9255,"threads_started":1}
I20260811 09:15:15.672674 31060 tablet_service.cc:1467] Processing CreateTablet for tablet c05fbc0c58be4e52ba7bdb421b9d0f4b (DEFAULT_TABLE table=test-table [id=d756bece07d141e7b761fa2589f8683e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:15.672555 31299 tablet_service.cc:1467] Processing CreateTablet for tablet c05fbc0c58be4e52ba7bdb421b9d0f4b (DEFAULT_TABLE table=test-table [id=d756bece07d141e7b761fa2589f8683e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:15.674074 31299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05fbc0c58be4e52ba7bdb421b9d0f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:15.674074 31060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05fbc0c58be4e52ba7bdb421b9d0f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:15.674831 30844 tablet_service.cc:1467] Processing CreateTablet for tablet c05fbc0c58be4e52ba7bdb421b9d0f4b (DEFAULT_TABLE table=test-table [id=d756bece07d141e7b761fa2589f8683e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:15.675873 30844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05fbc0c58be4e52ba7bdb421b9d0f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:15.686707 31456 tablet_bootstrap.cc:492] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: Bootstrap starting.
I20260811 09:15:15.688565 31457 tablet_bootstrap.cc:492] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: Bootstrap starting.
I20260811 09:15:15.691004 31456 tablet_bootstrap.cc:654] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:15.691991 31457 tablet_bootstrap.cc:654] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:15.692282 31456 log.cc:824] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:15.693252 31457 log.cc:824] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:15.694301 31458 tablet_bootstrap.cc:492] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: Bootstrap starting.
I20260811 09:15:15.695607 31456 tablet_bootstrap.cc:492] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: No bootstrap required, opened a new log
I20260811 09:15:15.695900 31456 ts_tablet_manager.cc:1397] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 09:15:15.697747 31458 tablet_bootstrap.cc:654] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:15.698395 31457 tablet_bootstrap.cc:492] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: No bootstrap required, opened a new log
I20260811 09:15:15.698732 31457 ts_tablet_manager.cc:1397] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260811 09:15:15.698951 31458 log.cc:824] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:15.703362 31458 tablet_bootstrap.cc:492] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: No bootstrap required, opened a new log
I20260811 09:15:15.703652 31458 ts_tablet_manager.cc:1397] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260811 09:15:15.705962 31456 raft_consensus.cc:348] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } }
I20260811 09:15:15.706182 31457 raft_consensus.cc:348] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } }
I20260811 09:15:15.706398 31456 raft_consensus.cc:374] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:15.706487 31456 raft_consensus.cc:729] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17f8ccef9f644f4a3d8a570baf6561d, State: Initialized, Role: FOLLOWER
I20260811 09:15:15.706545 31457 raft_consensus.cc:374] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:15.706668 31457 raft_consensus.cc:729] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f560081c29641c88dad14da789187a5, State: Initialized, Role: FOLLOWER
I20260811 09:15:15.707129 31456 consensus_queue.cc:260] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } }
I20260811 09:15:15.707228 31457 consensus_queue.cc:260] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } }
I20260811 09:15:15.708911 31456 ts_tablet_manager.cc:1428] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260811 09:15:15.711179 31457 ts_tablet_manager.cc:1428] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: Time spent starting tablet: real 0.012s user 0.004s sys 0.007s
I20260811 09:15:15.715252 31458 raft_consensus.cc:348] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } }
I20260811 09:15:15.715583 31458 raft_consensus.cc:374] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:15.715679 31458 raft_consensus.cc:729] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e3c5642bc4147eea8533ff8b62add6b, State: Initialized, Role: FOLLOWER
I20260811 09:15:15.716190 31458 consensus_queue.cc:260] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } }
I20260811 09:15:15.717717 31458 ts_tablet_manager.cc:1428] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: Time spent starting tablet: real 0.014s user 0.005s sys 0.008s
W20260811 09:15:15.742995 31185 tablet.cc:2369] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:15:15.763968 31463 raft_consensus.cc:482] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:15:15.764282 31463 raft_consensus.cc:504] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } }
I20260811 09:15:15.766011 31463 leader_election.cc:288] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683), 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947)
W20260811 09:15:15.772681 31412 tablet.cc:2369] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:15:15.773083 31449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.766463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:43675 (local address 127.17.16.67:35683)
0811 09:15:15.766670 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:15.766675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:15.767314 (+ 639us) server_negotiation.cc:407] Connection header received
0811 09:15:15.767483 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:15.767487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:15.767569 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:15.767647 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:15.768258 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.769219 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.771225 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.772091 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.772216 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.772267 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:15.772692 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 09:15:15.772821 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:15:15.773193 31468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.766469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:35683 (local address 127.17.16.65:43675)
0811 09:15:15.767198 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:15.767214 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:15.767331 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:15.767697 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:15.767702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:15.767729 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:15.768099 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:15.768113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.769343 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.769348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:15.771077 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:15.771090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.772297 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.772303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:15.772460 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.772495 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:15:15.772645 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 09:15:15.772770 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":45,"thread_start_us":444,"threads_started":1}
I20260811 09:15:15.773344 31455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.766422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:50923 (local address 127.17.16.66:33947)
0811 09:15:15.766860 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:15.766866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:15.767892 (+ 1026us) server_negotiation.cc:407] Connection header received
0811 09:15:15.768042 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:15.768047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:15.768138 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:15.768221 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:15.768795 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.769618 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.771536 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.772359 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.772509 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.772586 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:15.772984 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 09:15:15.773114 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260811 09:15:15.773936 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.766525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:33947 (local address 127.17.16.65:50923)
0811 09:15:15.767790 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:15.767805 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:15.767923 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:15.768252 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:15.768256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:15.768280 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:15.768646 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:15.768659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.769752 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.769775 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:15.771383 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:15.771398 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.772581 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.772588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:15.772751 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.772784 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:15:15.773516 (+ 732us) client_negotiation.cc:240] Negotiation successful
0811 09:15:15.773658 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":16,"thread_start_us":953,"threads_started":1}
I20260811 09:15:15.774401 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05fbc0c58be4e52ba7bdb421b9d0f4b" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3c5642bc4147eea8533ff8b62add6b" is_pre_election: true
I20260811 09:15:15.774887 31340 raft_consensus.cc:2454] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 0.
I20260811 09:15:15.774793 31103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05fbc0c58be4e52ba7bdb421b9d0f4b" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f560081c29641c88dad14da789187a5" is_pre_election: true
I20260811 09:15:15.775235 31103 raft_consensus.cc:2454] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 0.
I20260811 09:15:15.775519 30766 leader_election.cc:302] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e3c5642bc4147eea8533ff8b62add6b, a17f8ccef9f644f4a3d8a570baf6561d; no voters:
I20260811 09:15:15.775897 31463 raft_consensus.cc:2781] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:15:15.776041 31463 raft_consensus.cc:482] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:15:15.776151 31463 raft_consensus.cc:3037] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:15.778518 31463 raft_consensus.cc:504] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } }
I20260811 09:15:15.779170 31463 leader_election.cc:288] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 election: Requested vote from peers 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683), 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947)
I20260811 09:15:15.779510 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05fbc0c58be4e52ba7bdb421b9d0f4b" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3c5642bc4147eea8533ff8b62add6b"
I20260811 09:15:15.779623 31103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05fbc0c58be4e52ba7bdb421b9d0f4b" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f560081c29641c88dad14da789187a5"
I20260811 09:15:15.779697 31340 raft_consensus.cc:3037] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:15.779816 31103 raft_consensus.cc:3037] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:15.781927 31103 raft_consensus.cc:2454] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 1.
I20260811 09:15:15.781934 31340 raft_consensus.cc:2454] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 1.
I20260811 09:15:15.782322 30766 leader_election.cc:302] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e3c5642bc4147eea8533ff8b62add6b, a17f8ccef9f644f4a3d8a570baf6561d; no voters:
I20260811 09:15:15.782598 31463 raft_consensus.cc:2781] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:15.783057 31463 raft_consensus.cc:686] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 1 LEADER]: Becoming Leader. State: Replica: a17f8ccef9f644f4a3d8a570baf6561d, State: Running, Role: LEADER
I20260811 09:15:15.783515 31463 consensus_queue.cc:237] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } }
I20260811 09:15:15.787719 30617 catalog_manager.cc:5495] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d reported cstate change: term changed from 0 to 1, leader changed from <none> to a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65). New cstate: current_term: 1 leader_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260811 09:15:15.826030 31454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:15.819866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58674 (local address 127.17.16.65:41163)
0811 09:15:15.820069 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:15.820075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:15.820097 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:15:15.820279 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:15.820284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:15.820397 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:15.820579 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:15:15.820875 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.821959 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.823322 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:15.823989 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:15.824161 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:15.824625 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:15:15.825341 (+ 716us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:15:15.825455 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:15.825659 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 09:15:15.825751 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:15:15.834122 31103 raft_consensus.cc:1261] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Refusing update from remote peer a17f8ccef9f644f4a3d8a570baf6561d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:15.834240 31340 raft_consensus.cc:1261] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Refusing update from remote peer a17f8ccef9f644f4a3d8a570baf6561d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:15.835088 31471 consensus_queue.cc:1035] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:15.835434 31463 consensus_queue.cc:1035] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:15.860263 30623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:15.860492 30623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:15.864126 30623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4: Sending DeleteTablet for 3 replicas of tablet c05fbc0c58be4e52ba7bdb421b9d0f4b
I20260811 09:15:15.865350 30844 tablet_service.cc:1514] Processing DeleteTablet for tablet c05fbc0c58be4e52ba7bdb421b9d0f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:15 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:58668
I20260811 09:15:15.865341 31299 tablet_service.cc:1514] Processing DeleteTablet for tablet c05fbc0c58be4e52ba7bdb421b9d0f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:15 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:40740
I20260811 09:15:15.865479 31060 tablet_service.cc:1514] Processing DeleteTablet for tablet c05fbc0c58be4e52ba7bdb421b9d0f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:15 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:53794
I20260811 09:15:15.868633 30617 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:15:15.869331 30617 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:15:15.877666 31490 tablet_replica.cc:330] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: stopping tablet replica
I20260811 09:15:15.877914 31491 tablet_replica.cc:330] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: stopping tablet replica
I20260811 09:15:15.878470 31491 raft_consensus.cc:2229] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:15.878633 31059 tablet_service.cc:1467] Processing CreateTablet for tablet d731ac90fe044df7a63a2220a9a02995 (DEFAULT_TABLE table=test-table [id=6d453c961dfd4cb79bf7fb0c97c36a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:15.878742 31490 raft_consensus.cc:2229] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:15:15.879092 31491 raft_consensus.cc:2258] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:15.879307 31059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d731ac90fe044df7a63a2220a9a02995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:15.879908 31486 tablet_replica.cc:330] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: stopping tablet replica
I20260811 09:15:15.879863 31490 raft_consensus.cc:2258] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:15.880265 31486 raft_consensus.cc:2229] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:15.880713 31486 raft_consensus.cc:2258] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:15.877568 30843 tablet_service.cc:1467] Processing CreateTablet for tablet d731ac90fe044df7a63a2220a9a02995 (DEFAULT_TABLE table=test-table [id=6d453c961dfd4cb79bf7fb0c97c36a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:15.881337 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d731ac90fe044df7a63a2220a9a02995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:15.882685 31491 ts_tablet_manager.cc:1905] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:15.887544 31457 tablet_bootstrap.cc:492] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5: Bootstrap starting.
I20260811 09:15:15.888722 31299 tablet_service.cc:1467] Processing CreateTablet for tablet d731ac90fe044df7a63a2220a9a02995 (DEFAULT_TABLE table=test-table [id=6d453c961dfd4cb79bf7fb0c97c36a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:15.889127 31457 tablet_bootstrap.cc:654] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:15.889164 31299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d731ac90fe044df7a63a2220a9a02995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:15.890293 31491 ts_tablet_manager.cc:1918] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:15.890480 31491 log.cc:1196] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/c05fbc0c58be4e52ba7bdb421b9d0f4b
I20260811 09:15:15.891085 31491 ts_tablet_manager.cc:1939] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2f560081c29641c88dad14da789187a5: Deleting consensus metadata
I20260811 09:15:15.892530 31486 ts_tablet_manager.cc:1905] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:15.892818 30590 catalog_manager.cc:4841] TS 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947): tablet c05fbc0c58be4e52ba7bdb421b9d0f4b (table test-table [id=d756bece07d141e7b761fa2589f8683e]) successfully deleted
I20260811 09:15:15.893884 31456 tablet_bootstrap.cc:492] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: Bootstrap starting.
I20260811 09:15:15.895491 31490 ts_tablet_manager.cc:1905] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:15.895663 31456 tablet_bootstrap.cc:654] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:15.901751 31458 tablet_bootstrap.cc:492] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b: Bootstrap starting.
I20260811 09:15:15.903450 31458 tablet_bootstrap.cc:654] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:15.903589 31490 ts_tablet_manager.cc:1918] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:15.903846 31490 log.cc:1196] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/c05fbc0c58be4e52ba7bdb421b9d0f4b
I20260811 09:15:15.904482 31490 ts_tablet_manager.cc:1939] T c05fbc0c58be4e52ba7bdb421b9d0f4b P a17f8ccef9f644f4a3d8a570baf6561d: Deleting consensus metadata
I20260811 09:15:15.906453 30593 catalog_manager.cc:4841] TS a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65:41163): tablet c05fbc0c58be4e52ba7bdb421b9d0f4b (table test-table [id=d756bece07d141e7b761fa2589f8683e]) successfully deleted
I20260811 09:15:15.910017 31486 ts_tablet_manager.cc:1918] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:15.910239 31486 log.cc:1196] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/c05fbc0c58be4e52ba7bdb421b9d0f4b
I20260811 09:15:15.910992 31486 ts_tablet_manager.cc:1939] T c05fbc0c58be4e52ba7bdb421b9d0f4b P 2e3c5642bc4147eea8533ff8b62add6b: Deleting consensus metadata
I20260811 09:15:15.911810 31457 tablet_bootstrap.cc:492] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5: No bootstrap required, opened a new log
I20260811 09:15:15.912035 31457 ts_tablet_manager.cc:1397] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5: Time spent bootstrapping tablet: real 0.025s user 0.011s sys 0.011s
I20260811 09:15:15.912155 31456 tablet_bootstrap.cc:492] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: No bootstrap required, opened a new log
I20260811 09:15:15.912165 31458 tablet_bootstrap.cc:492] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b: No bootstrap required, opened a new log
I20260811 09:15:15.912303 31458 ts_tablet_manager.cc:1397] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 09:15:15.912372 31456 ts_tablet_manager.cc:1397] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.008s
I20260811 09:15:15.912843 30595 catalog_manager.cc:4841] TS 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683): tablet c05fbc0c58be4e52ba7bdb421b9d0f4b (table test-table [id=d756bece07d141e7b761fa2589f8683e]) successfully deleted
I20260811 09:15:15.912932 31457 raft_consensus.cc:348] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:15.913160 31458 raft_consensus.cc:348] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:15.913270 31457 raft_consensus.cc:374] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:15.913188 31456 raft_consensus.cc:348] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:15.913352 31458 raft_consensus.cc:374] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:15.913360 31457 raft_consensus.cc:729] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f560081c29641c88dad14da789187a5, State: Initialized, Role: FOLLOWER
I20260811 09:15:15.913391 31456 raft_consensus.cc:374] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:15.913517 31456 raft_consensus.cc:729] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17f8ccef9f644f4a3d8a570baf6561d, State: Initialized, Role: FOLLOWER
I20260811 09:15:15.913460 31458 raft_consensus.cc:729] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e3c5642bc4147eea8533ff8b62add6b, State: Initialized, Role: FOLLOWER
I20260811 09:15:15.913627 31457 consensus_queue.cc:260] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:15.913803 31456 consensus_queue.cc:260] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:15.913815 31458 consensus_queue.cc:260] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:15.914225 31457 ts_tablet_manager.cc:1428] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 09:15:15.914532 31456 ts_tablet_manager.cc:1428] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 09:15:15.915161 31458 ts_tablet_manager.cc:1428] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20260811 09:15:15.955880 30956 tablet.cc:2369] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:15:16.148962 31471 raft_consensus.cc:482] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:15:16.149271 31471 raft_consensus.cc:504] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.150424 31471 leader_election.cc:288] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947), 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683)
I20260811 09:15:16.150995 31103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d731ac90fe044df7a63a2220a9a02995" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f560081c29641c88dad14da789187a5" is_pre_election: true
I20260811 09:15:16.151072 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d731ac90fe044df7a63a2220a9a02995" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3c5642bc4147eea8533ff8b62add6b" is_pre_election: true
I20260811 09:15:16.151239 31103 raft_consensus.cc:2454] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 0.
I20260811 09:15:16.151363 31340 raft_consensus.cc:2454] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 0.
I20260811 09:15:16.151717 30764 leader_election.cc:302] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f560081c29641c88dad14da789187a5, a17f8ccef9f644f4a3d8a570baf6561d; no voters:
I20260811 09:15:16.152101 31471 raft_consensus.cc:2781] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:15:16.152230 31471 raft_consensus.cc:482] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:15:16.152366 31471 raft_consensus.cc:3037] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:16.154680 31471 raft_consensus.cc:504] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.155238 31471 leader_election.cc:288] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 election: Requested vote from peers 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947), 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683)
I20260811 09:15:16.155686 31103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d731ac90fe044df7a63a2220a9a02995" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f560081c29641c88dad14da789187a5"
I20260811 09:15:16.155820 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d731ac90fe044df7a63a2220a9a02995" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3c5642bc4147eea8533ff8b62add6b"
I20260811 09:15:16.155894 31103 raft_consensus.cc:3037] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:16.156009 31340 raft_consensus.cc:3037] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:16.158842 31340 raft_consensus.cc:2454] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 1.
I20260811 09:15:16.158842 31103 raft_consensus.cc:2454] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 1.
I20260811 09:15:16.159236 30764 leader_election.cc:302] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f560081c29641c88dad14da789187a5, a17f8ccef9f644f4a3d8a570baf6561d; no voters:
I20260811 09:15:16.159479 31471 raft_consensus.cc:2781] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:16.159649 31471 raft_consensus.cc:686] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 LEADER]: Becoming Leader. State: Replica: a17f8ccef9f644f4a3d8a570baf6561d, State: Running, Role: LEADER
I20260811 09:15:16.159926 31471 consensus_queue.cc:237] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.162773 30623 catalog_manager.cc:5495] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d reported cstate change: term changed from 0 to 1, leader changed from <none> to a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65). New cstate: current_term: 1 leader_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } health_report { overall_health: UNKNOWN } } }
Aug 11 09:15:16 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439712, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.126@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:16.233723 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.219399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48486 (local address 127.17.16.126:46579)
0811 09:15:16.220250 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:16.220255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:16.220273 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:15:16.220327 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:16.220332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:16.220399 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:16.220491 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:16.221619 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.222401 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.223658 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.224154 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.224262 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.228093 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:16.228098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:16.228101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:16.228130 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:16.230652 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:16.231536 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:16.231539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:16.231541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:16.231652 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:16.232102 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:16.232106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:16.232108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:16.232363 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:16.232483 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:16.233096 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 09:15:16.233446 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":171,"server-negotiator.queue_time_us":710,"thread_start_us":565,"threads_started":1}
I20260811 09:15:16.233958 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.219283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46579 (local address 127.0.0.1:48486)
0811 09:15:16.219928 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:16.219950 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:16.220117 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:16.220564 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:16.220568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:16.221281 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:16.221483 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.221494 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.222541 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.222545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.223523 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.223537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.224309 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.224315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.224480 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.225134 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:16.225158 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:16.227888 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:16.230799 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:16.230813 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:16.230823 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:16.231415 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:16.231759 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:16.231764 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:16.231768 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:16.231989 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:16.232510 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:16.232519 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:16.232738 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 09:15:16.233018 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 09:15:16.233346 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":325,"threads_started":1}
I20260811 09:15:16.239253 30623 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:48486:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 09:15:16.240237 30623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:15:16.247017 31059 tablet_service.cc:1467] Processing CreateTablet for tablet c391043a329f4082a15e787732ee6341 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6862c46e2ecd42528fa9c4f48adf7ae1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:15:16.246997 30843 tablet_service.cc:1467] Processing CreateTablet for tablet c391043a329f4082a15e787732ee6341 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6862c46e2ecd42528fa9c4f48adf7ae1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:15:16.247121 31299 tablet_service.cc:1467] Processing CreateTablet for tablet c391043a329f4082a15e787732ee6341 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6862c46e2ecd42528fa9c4f48adf7ae1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:15:16.247610 31059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c391043a329f4082a15e787732ee6341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:16.247624 31299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c391043a329f4082a15e787732ee6341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:16.247673 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c391043a329f4082a15e787732ee6341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:16.254776 31457 tablet_bootstrap.cc:492] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: Bootstrap starting.
I20260811 09:15:16.255576 31458 tablet_bootstrap.cc:492] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: Bootstrap starting.
I20260811 09:15:16.256522 31457 tablet_bootstrap.cc:654] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:16.256705 31456 tablet_bootstrap.cc:492] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: Bootstrap starting.
I20260811 09:15:16.257283 31458 tablet_bootstrap.cc:654] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:16.258693 31456 tablet_bootstrap.cc:654] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:16.266095 31456 tablet_bootstrap.cc:492] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: No bootstrap required, opened a new log
I20260811 09:15:16.266357 31456 ts_tablet_manager.cc:1397] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 09:15:16.267234 31458 tablet_bootstrap.cc:492] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: No bootstrap required, opened a new log
I20260811 09:15:16.267401 31458 ts_tablet_manager.cc:1397] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260811 09:15:16.267225 31456 raft_consensus.cc:348] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.267516 31456 raft_consensus.cc:374] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:16.267622 31456 raft_consensus.cc:729] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17f8ccef9f644f4a3d8a570baf6561d, State: Initialized, Role: FOLLOWER
I20260811 09:15:16.268077 31458 raft_consensus.cc:348] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.268308 31458 raft_consensus.cc:374] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:16.268388 31458 raft_consensus.cc:729] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e3c5642bc4147eea8533ff8b62add6b, State: Initialized, Role: FOLLOWER
I20260811 09:15:16.268537 31456 consensus_queue.cc:260] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.268746 31458 consensus_queue.cc:260] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.269277 31456 ts_tablet_manager.cc:1428] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 09:15:16.269119 31471 tablet_replica.cc:439] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } }
I20260811 09:15:16.269260 31467 tablet_replica.cc:439] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } }
I20260811 09:15:16.269495 31471 tablet_replica.cc:442] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:15:16.269550 31467 tablet_replica.cc:442] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:15:16.270493 31458 ts_tablet_manager.cc:1428] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 09:15:16.271829 31457 tablet_bootstrap.cc:492] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: No bootstrap required, opened a new log
I20260811 09:15:16.272055 31457 ts_tablet_manager.cc:1397] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.010s
I20260811 09:15:16.273321 31457 raft_consensus.cc:348] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.273597 31457 raft_consensus.cc:374] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:16.273728 31457 raft_consensus.cc:729] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f560081c29641c88dad14da789187a5, State: Initialized, Role: FOLLOWER
I20260811 09:15:16.273998 31457 consensus_queue.cc:260] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.274555 31457 ts_tablet_manager.cc:1428] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 09:15:16.274581 31466 tablet_replica.cc:439] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } }
I20260811 09:15:16.274919 31466 tablet_replica.cc:442] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:15:16.460780 31466 raft_consensus.cc:482] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:15:16.461050 31466 raft_consensus.cc:504] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.462862 31466 leader_election.cc:288] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65:41163), 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683)
I20260811 09:15:16.477180 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.463155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:41163 (local address 127.17.16.66:43929)
0811 09:15:16.469716 (+ 6561us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:16.469737 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:16.469904 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:16.470423 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:16.470428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:16.470458 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:16.470818 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.470832 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.472281 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.472286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.474717 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.474735 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.476299 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.476304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.476441 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.476479 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 09:15:16.476579 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 09:15:16.476719 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6429,"mutex_wait_us":92,"thread_start_us":6564,"threads_started":1}
I20260811 09:15:16.477121 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.463443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:43929 (local address 127.17.16.65:41163)
0811 09:15:16.469277 (+ 5834us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:16.469285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:16.469996 (+ 711us) server_negotiation.cc:407] Connection header received
0811 09:15:16.470148 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:16.470154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:16.470237 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:16.470388 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:16.471041 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.472142 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.474876 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.476112 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.476277 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.476353 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:16.476644 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 09:15:16.476792 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5690,"thread_start_us":5577,"threads_started":1}
I20260811 09:15:16.478381 31518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.463157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:35683 (local address 127.17.16.66:51391)
0811 09:15:16.470084 (+ 6927us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:16.470103 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:16.470223 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:16.470640 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:16.470645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:16.470668 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:16.471076 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.471108 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.472197 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.472204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.474717 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.474735 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.477622 (+ 2887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.477629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.477820 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.477859 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 09:15:16.477953 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 09:15:16.478078 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6737,"mutex_wait_us":605,"thread_start_us":5761,"threads_started":1}
I20260811 09:15:16.478183 30879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c391043a329f4082a15e787732ee6341" candidate_uuid: "2f560081c29641c88dad14da789187a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" is_pre_election: true
I20260811 09:15:16.478420 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.463315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:51391 (local address 127.17.16.67:35683)
0811 09:15:16.470346 (+ 7031us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:16.470352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:16.470370 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:15:16.470441 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:16.470446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:16.470517 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:16.470604 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:16.471285 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.472053 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.476446 (+ 4393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.477405 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.477563 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.477631 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:16.477988 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 09:15:16.478144 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6920,"thread_start_us":6828,"threads_started":1}
I20260811 09:15:16.478637 30879 raft_consensus.cc:2454] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f560081c29641c88dad14da789187a5 in term 0.
I20260811 09:15:16.479014 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c391043a329f4082a15e787732ee6341" candidate_uuid: "2f560081c29641c88dad14da789187a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3c5642bc4147eea8533ff8b62add6b" is_pre_election: true
I20260811 09:15:16.479182 30983 leader_election.cc:302] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f560081c29641c88dad14da789187a5, a17f8ccef9f644f4a3d8a570baf6561d; no voters:
I20260811 09:15:16.479311 31340 raft_consensus.cc:2454] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f560081c29641c88dad14da789187a5 in term 0.
I20260811 09:15:16.479581 31466 raft_consensus.cc:2781] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:15:16.479728 31466 raft_consensus.cc:482] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:15:16.479843 31466 raft_consensus.cc:3037] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:16.482384 31466 raft_consensus.cc:504] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.482980 31466 leader_election.cc:288] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [CANDIDATE]: Term 1 election: Requested vote from peers a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65:41163), 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683)
I20260811 09:15:16.483413 30879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c391043a329f4082a15e787732ee6341" candidate_uuid: "2f560081c29641c88dad14da789187a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17f8ccef9f644f4a3d8a570baf6561d"
I20260811 09:15:16.483462 31340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c391043a329f4082a15e787732ee6341" candidate_uuid: "2f560081c29641c88dad14da789187a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3c5642bc4147eea8533ff8b62add6b"
I20260811 09:15:16.483650 30879 raft_consensus.cc:3037] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:16.483676 31340 raft_consensus.cc:3037] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:16.486677 31340 raft_consensus.cc:2454] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f560081c29641c88dad14da789187a5 in term 1.
I20260811 09:15:16.486694 30879 raft_consensus.cc:2454] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f560081c29641c88dad14da789187a5 in term 1.
I20260811 09:15:16.487114 30983 leader_election.cc:302] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2e3c5642bc4147eea8533ff8b62add6b, 2f560081c29641c88dad14da789187a5, a17f8ccef9f644f4a3d8a570baf6561d; no voters:
I20260811 09:15:16.487444 31466 raft_consensus.cc:2781] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:16.487812 31466 raft_consensus.cc:686] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f560081c29641c88dad14da789187a5, State: Running, Role: LEADER
I20260811 09:15:16.488152 31466 consensus_queue.cc:237] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.490127 31523 tablet_replica.cc:439] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: TxnStatusTablet state changed. Reason: New leader 2f560081c29641c88dad14da789187a5. Latest consensus state: current_term: 1 leader_uuid: "2f560081c29641c88dad14da789187a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } }
I20260811 09:15:16.490559 31523 tablet_replica.cc:442] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:15:16.491371 31528 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 09:15:16.491575 31528 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 09:15:16.492004 30623 catalog_manager.cc:5495] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f560081c29641c88dad14da789187a5 (127.17.16.66). New cstate: current_term: 1 leader_uuid: "2f560081c29641c88dad14da789187a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } health_report { overall_health: UNKNOWN } } }
I20260811 09:15:16.561606 31531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.555228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.17.16.66:33947)
0811 09:15:16.555829 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:16.555834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:16.555853 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:15:16.555931 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:16.555934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:16.556008 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:16.556144 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:15:16.556541 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.557465 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.559365 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.559807 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.559893 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.560431 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:15:16.561021 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:15:16.561128 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:16.561259 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 09:15:16.561323 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":344,"threads_started":1}
I20260811 09:15:16.561692 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.555217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:33947 (local address 127.0.0.1:53806)
0811 09:15:16.555495 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:16.555516 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:16.555656 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:16.556181 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:16.556186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:16.556200 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:15:16.556392 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.556403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.557594 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.557600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.559239 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.559255 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.559944 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.559952 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.560158 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.560229 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:15:16.561118 (+ 889us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:15:16.561141 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:15:16.561229 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 09:15:16.561380 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 09:15:16.565963 31340 raft_consensus.cc:1261] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Refusing update from remote peer 2f560081c29641c88dad14da789187a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:16.566347 30879 raft_consensus.cc:1261] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Refusing update from remote peer 2f560081c29641c88dad14da789187a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:16.566720 31523 consensus_queue.cc:1035] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:16.567042 31466 consensus_queue.cc:1035] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:16.570689 31467 tablet_replica.cc:439] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: TxnStatusTablet state changed. Reason: New leader 2f560081c29641c88dad14da789187a5. Latest consensus state: current_term: 1 leader_uuid: "2f560081c29641c88dad14da789187a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } }
I20260811 09:15:16.571007 31467 tablet_replica.cc:442] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:15:16.572860 31532 mvcc.cc:205] Tried to move back new op lower bound from 7317257079047282688 to 7317257078739230720. Current Snapshot: MvccSnapshot[applied={T|T < 7317257079047282688}]
I20260811 09:15:16.574710 31467 tablet_replica.cc:439] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f560081c29641c88dad14da789187a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } }
I20260811 09:15:16.574957 31467 tablet_replica.cc:442] T c391043a329f4082a15e787732ee6341 P 2e3c5642bc4147eea8533ff8b62add6b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:15:16.576563 31466 tablet_replica.cc:439] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f560081c29641c88dad14da789187a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } }
I20260811 09:15:16.576864 31466 tablet_replica.cc:442] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:15:16.582069 31471 tablet_replica.cc:439] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: TxnStatusTablet state changed. Reason: New leader 2f560081c29641c88dad14da789187a5. Latest consensus state: current_term: 1 leader_uuid: "2f560081c29641c88dad14da789187a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } }
I20260811 09:15:16.582357 31471 tablet_replica.cc:442] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:15:16.583688 31466 tablet_replica.cc:439] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f560081c29641c88dad14da789187a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } }
I20260811 09:15:16.583871 31466 tablet_replica.cc:442] T c391043a329f4082a15e787732ee6341 P 2f560081c29641c88dad14da789187a5: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:15:16.586380 31471 tablet_replica.cc:439] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f560081c29641c88dad14da789187a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } } }
I20260811 09:15:16.586616 31471 tablet_replica.cc:442] T c391043a329f4082a15e787732ee6341 P a17f8ccef9f644f4a3d8a570baf6561d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:15:16.600664 31531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.594800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:40689 (local address 127.17.16.66:33947)
0811 09:15:16.594994 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:16.595000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:16.595394 (+ 394us) server_negotiation.cc:407] Connection header received
0811 09:15:16.595554 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:16.595560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:16.595646 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:16.595739 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:15:16.596172 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.597326 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.598621 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.599321 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.599442 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.599884 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:15:16.600019 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:15:16.600085 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:16.600331 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 09:15:16.600398 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:15:16.601269 31546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.594635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:33947 (local address 127.17.16.65:40689)
0811 09:15:16.595309 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:16.595325 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:16.595442 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:16.595743 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:16.595746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:16.595755 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:15:16.596018 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.596025 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.597443 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.597447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.598487 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.598499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.599453 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.599457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.599591 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.599671 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:15:16.600186 (+ 515us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:15:16.600197 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 09:15:16.600835 (+ 638us) client_negotiation.cc:240] Negotiation successful
0811 09:15:16.600993 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":359,"threads_started":1}
I20260811 09:15:16.610785 31471 consensus_queue.cc:1035] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:16.611320 31471 consensus_queue.cc:1035] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:16.615574 31535 mvcc.cc:205] Tried to move back new op lower bound from 7317257079234990080 to 7317257077393543168. Current Snapshot: MvccSnapshot[applied={T|T < 7317257079234990080}]
I20260811 09:15:16.664772 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.644729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:36587 (local address 127.17.16.65:41163)
0811 09:15:16.644953 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:16.644986 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:16.657960 (+ 12974us) server_negotiation.cc:407] Connection header received
0811 09:15:16.658081 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:16.658088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:16.658201 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:16.658335 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:15:16.659039 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.660198 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.662464 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.663121 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.663265 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.663941 (+ 676us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:15:16.664095 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:15:16.664162 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:16.664378 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 09:15:16.664482 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 09:15:16.665876 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.644689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:41163 (local address 127.17.16.66:36587)
0811 09:15:16.657114 (+ 12425us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:16.657136 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:16.657296 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:16.658356 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:16.658362 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:16.658377 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:15:16.658823 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.658838 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.660392 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.660397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.662301 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:16.662318 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.663415 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.663422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:16.663616 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.663707 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:15:16.664244 (+ 537us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:15:16.664255 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 09:15:16.665288 (+ 1033us) client_negotiation.cc:240] Negotiation successful
0811 09:15:16.665446 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12152,"thread_start_us":12684,"threads_started":1}
I20260811 09:15:16.706306 30623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:16.706707 30623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:16.716231 30623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4: Sending DeleteTablet for 3 replicas of tablet d731ac90fe044df7a63a2220a9a02995
I20260811 09:15:16.717235 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet d731ac90fe044df7a63a2220a9a02995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:16 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:58668
I20260811 09:15:16.718106 31566 tablet_replica.cc:330] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: stopping tablet replica
I20260811 09:15:16.718282 31299 tablet_service.cc:1514] Processing DeleteTablet for tablet d731ac90fe044df7a63a2220a9a02995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:16 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:40740
I20260811 09:15:16.718325 31566 raft_consensus.cc:2229] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:15:16.718781 31566 raft_consensus.cc:2258] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:16.719383 31567 tablet_replica.cc:330] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b: stopping tablet replica
I20260811 09:15:16.719655 31567 raft_consensus.cc:2229] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:16.719969 31567 raft_consensus.cc:2258] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:16.719990 31566 ts_tablet_manager.cc:1905] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:16.719660 31059 tablet_service.cc:1514] Processing DeleteTablet for tablet d731ac90fe044df7a63a2220a9a02995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:16 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:53794
I20260811 09:15:16.722726 31567 ts_tablet_manager.cc:1905] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:16.727241 31566 ts_tablet_manager.cc:1918] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:15:16.727386 31566 log.cc:1196] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/d731ac90fe044df7a63a2220a9a02995
I20260811 09:15:16.727841 31566 ts_tablet_manager.cc:1939] T d731ac90fe044df7a63a2220a9a02995 P a17f8ccef9f644f4a3d8a570baf6561d: Deleting consensus metadata
I20260811 09:15:16.729069 30593 catalog_manager.cc:4841] TS a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65:41163): tablet d731ac90fe044df7a63a2220a9a02995 (table test-table [id=6d453c961dfd4cb79bf7fb0c97c36a6c]) successfully deleted
I20260811 09:15:16.730022 31567 ts_tablet_manager.cc:1918] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:15:16.730213 31567 log.cc:1196] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/d731ac90fe044df7a63a2220a9a02995
I20260811 09:15:16.730698 31567 ts_tablet_manager.cc:1939] T d731ac90fe044df7a63a2220a9a02995 P 2e3c5642bc4147eea8533ff8b62add6b: Deleting consensus metadata
I20260811 09:15:16.731742 30595 catalog_manager.cc:4841] TS 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683): tablet d731ac90fe044df7a63a2220a9a02995 (table test-table [id=6d453c961dfd4cb79bf7fb0c97c36a6c]) successfully deleted
I20260811 09:15:16.739123 31568 tablet_replica.cc:330] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5: stopping tablet replica
I20260811 09:15:16.739512 31568 raft_consensus.cc:2229] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:16.739907 31568 raft_consensus.cc:2258] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:16.741318 31568 ts_tablet_manager.cc:1905] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:16.749127 31568 ts_tablet_manager.cc:1918] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:15:16.749287 31568 log.cc:1196] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/d731ac90fe044df7a63a2220a9a02995
I20260811 09:15:16.749742 31568 ts_tablet_manager.cc:1939] T d731ac90fe044df7a63a2220a9a02995 P 2f560081c29641c88dad14da789187a5: Deleting consensus metadata
I20260811 09:15:16.751003 30590 catalog_manager.cc:4841] TS 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947): tablet d731ac90fe044df7a63a2220a9a02995 (table test-table [id=6d453c961dfd4cb79bf7fb0c97c36a6c]) successfully deleted
Aug 11 09:15:16 dist-test-slave-ncp9 krb5kdc[30539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:15:16 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:16 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:15:16.791014 31580 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:16.779987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.17.16.126:46579)
0811 09:15:16.780942 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:16.780948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:16.780968 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:15:16.781037 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:16.781041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:16.781104 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:16.781201 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:16.782573 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.783368 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.784654 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.785130 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.785244 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.790697 (+ 5453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48502: BlockingRecv error: recv got EOF from 127.0.0.1:48502 (error 108)
Metrics: {"server-negotiator.queue_time_us":807,"thread_start_us":700,"threads_started":1}
Aug 11 09:15:16 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM
I20260811 09:15:16.808230 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:16.793668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48516 (local address 127.17.16.126:46579)
0811 09:15:16.793856 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:16.793862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:16.794133 (+ 271us) server_negotiation.cc:407] Connection header received
0811 09:15:16.794259 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:16.794264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:16.794341 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:16.794439 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:16.795560 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.796666 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.797952 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:16.798580 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:16.798712 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:16.802444 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:16.802451 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:16.802455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:16.802496 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:16.805759 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:16.806367 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:16.806372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:16.806376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:16.806561 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:16.806875 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:16.806880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:16.806883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:16.807247 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:16.807520 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:16.807645 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 09:15:16.807865 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:15:16.812307 30623 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:15:16.813136 30623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:15:16.821525 30843 tablet_service.cc:1467] Processing CreateTablet for tablet 64ecc9a319b34bd38da284f74652d006 (DEFAULT_TABLE table=test-table [id=02ba5d378072412aa34a268bd2494763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:16.821753 31299 tablet_service.cc:1467] Processing CreateTablet for tablet 64ecc9a319b34bd38da284f74652d006 (DEFAULT_TABLE table=test-table [id=02ba5d378072412aa34a268bd2494763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:16.822170 31299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64ecc9a319b34bd38da284f74652d006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:16.822182 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64ecc9a319b34bd38da284f74652d006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:16.825620 31059 tablet_service.cc:1467] Processing CreateTablet for tablet 64ecc9a319b34bd38da284f74652d006 (DEFAULT_TABLE table=test-table [id=02ba5d378072412aa34a268bd2494763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:16.826121 31059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64ecc9a319b34bd38da284f74652d006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:16.834141 31593 tablet_bootstrap.cc:492] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b: Bootstrap starting.
I20260811 09:15:16.835333 31592 tablet_bootstrap.cc:492] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d: Bootstrap starting.
I20260811 09:15:16.837000 31592 tablet_bootstrap.cc:654] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:16.837738 31593 tablet_bootstrap.cc:654] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:16.846133 31593 tablet_bootstrap.cc:492] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b: No bootstrap required, opened a new log
I20260811 09:15:16.846413 31593 ts_tablet_manager.cc:1397] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 09:15:16.847294 31593 raft_consensus.cc:348] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.847698 31593 raft_consensus.cc:374] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:16.847851 31593 raft_consensus.cc:729] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e3c5642bc4147eea8533ff8b62add6b, State: Initialized, Role: FOLLOWER
I20260811 09:15:16.848207 31593 consensus_queue.cc:260] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.849606 31604 tablet_bootstrap.cc:492] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5: Bootstrap starting.
I20260811 09:15:16.849735 31593 ts_tablet_manager.cc:1428] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 09:15:16.850337 31592 tablet_bootstrap.cc:492] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d: No bootstrap required, opened a new log
I20260811 09:15:16.850489 31592 ts_tablet_manager.cc:1397] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.011s
I20260811 09:15:16.851243 31592 raft_consensus.cc:348] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.851476 31604 tablet_bootstrap.cc:654] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:16.851504 31592 raft_consensus.cc:374] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:16.851605 31592 raft_consensus.cc:729] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17f8ccef9f644f4a3d8a570baf6561d, State: Initialized, Role: FOLLOWER
I20260811 09:15:16.851879 31592 consensus_queue.cc:260] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.852392 31592 ts_tablet_manager.cc:1428] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 09:15:16.855383 31604 tablet_bootstrap.cc:492] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5: No bootstrap required, opened a new log
I20260811 09:15:16.855557 31604 ts_tablet_manager.cc:1397] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 09:15:16.856487 31604 raft_consensus.cc:348] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.856761 31604 raft_consensus.cc:374] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:16.856941 31604 raft_consensus.cc:729] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f560081c29641c88dad14da789187a5, State: Initialized, Role: FOLLOWER
I20260811 09:15:16.857270 31604 consensus_queue.cc:260] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:16.857921 31604 ts_tablet_manager.cc:1428] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 09:15:17.098187 31466 raft_consensus.cc:482] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:15:17.098364 31466 raft_consensus.cc:504] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.098920 31466 leader_election.cc:288] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65:41163), 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683)
I20260811 09:15:17.099371 30878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ecc9a319b34bd38da284f74652d006" candidate_uuid: "2f560081c29641c88dad14da789187a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" is_pre_election: true
I20260811 09:15:17.099483 31339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ecc9a319b34bd38da284f74652d006" candidate_uuid: "2f560081c29641c88dad14da789187a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3c5642bc4147eea8533ff8b62add6b" is_pre_election: true
I20260811 09:15:17.099658 30878 raft_consensus.cc:2454] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f560081c29641c88dad14da789187a5 in term 0.
I20260811 09:15:17.099726 31339 raft_consensus.cc:2454] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f560081c29641c88dad14da789187a5 in term 0.
I20260811 09:15:17.100086 30983 leader_election.cc:302] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f560081c29641c88dad14da789187a5, a17f8ccef9f644f4a3d8a570baf6561d; no voters:
I20260811 09:15:17.100385 31466 raft_consensus.cc:2781] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:15:17.100492 31466 raft_consensus.cc:482] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:15:17.100577 31466 raft_consensus.cc:3037] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:17.102669 31466 raft_consensus.cc:504] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.103188 31466 leader_election.cc:288] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [CANDIDATE]: Term 1 election: Requested vote from peers a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65:41163), 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683)
I20260811 09:15:17.103650 30878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ecc9a319b34bd38da284f74652d006" candidate_uuid: "2f560081c29641c88dad14da789187a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a17f8ccef9f644f4a3d8a570baf6561d"
I20260811 09:15:17.103682 31339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64ecc9a319b34bd38da284f74652d006" candidate_uuid: "2f560081c29641c88dad14da789187a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3c5642bc4147eea8533ff8b62add6b"
I20260811 09:15:17.103869 31339 raft_consensus.cc:3037] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:17.103873 30878 raft_consensus.cc:3037] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:17.106725 31339 raft_consensus.cc:2454] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f560081c29641c88dad14da789187a5 in term 1.
I20260811 09:15:17.106784 30878 raft_consensus.cc:2454] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f560081c29641c88dad14da789187a5 in term 1.
I20260811 09:15:17.107157 30984 leader_election.cc:302] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e3c5642bc4147eea8533ff8b62add6b, 2f560081c29641c88dad14da789187a5; no voters:
I20260811 09:15:17.107426 31466 raft_consensus.cc:2781] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:17.107558 31466 raft_consensus.cc:686] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f560081c29641c88dad14da789187a5, State: Running, Role: LEADER
I20260811 09:15:17.107816 31466 consensus_queue.cc:237] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.110349 30613 catalog_manager.cc:5495] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f560081c29641c88dad14da789187a5 (127.17.16.66). New cstate: current_term: 1 leader_uuid: "2f560081c29641c88dad14da789187a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } health_report { overall_health: UNKNOWN } } }
I20260811 09:15:17.140439 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:17.135550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.17.16.66:33947)
0811 09:15:17.136040 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:17.136044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:17.136063 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:15:17.136122 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:17.136125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:17.136192 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:17.136281 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:15:17.136614 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:17.137391 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:17.138799 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:17.139281 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:17.139376 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:17.139762 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:15:17.139871 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:15:17.139935 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:17.140057 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 09:15:17.140132 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":286,"threads_started":1}
I20260811 09:15:17.143518 30878 raft_consensus.cc:1261] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Refusing update from remote peer 2f560081c29641c88dad14da789187a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:17.143517 31339 raft_consensus.cc:1261] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Refusing update from remote peer 2f560081c29641c88dad14da789187a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:17.144162 31557 consensus_queue.cc:1035] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:17.144493 31466 consensus_queue.cc:1035] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:17.158084 30623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:17.158318 30623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:17.162339 30623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4: Sending DeleteTablet for 3 replicas of tablet 64ecc9a319b34bd38da284f74652d006
I20260811 09:15:17.163323 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet 64ecc9a319b34bd38da284f74652d006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:17 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:58668
I20260811 09:15:17.163609 31566 tablet_replica.cc:330] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d: stopping tablet replica
I20260811 09:15:17.163589 31059 tablet_service.cc:1514] Processing DeleteTablet for tablet 64ecc9a319b34bd38da284f74652d006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:17 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:53794
I20260811 09:15:17.163753 31060 tablet_service.cc:1514] Processing DeleteTablet for tablet 64ecc9a319b34bd38da284f74652d006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:17 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:53794
I20260811 09:15:17.163805 31566 raft_consensus.cc:2229] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:17.163791 31299 tablet_service.cc:1514] Processing DeleteTablet for tablet 64ecc9a319b34bd38da284f74652d006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:17 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:40740
I20260811 09:15:17.164008 31566 raft_consensus.cc:2258] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:17.164038 31567 tablet_replica.cc:330] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b: stopping tablet replica
I20260811 09:15:17.164072 31568 tablet_replica.cc:330] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5: stopping tablet replica
I20260811 09:15:17.164218 31567 raft_consensus.cc:2229] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:17.164263 31568 raft_consensus.cc:2229] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:15:17.164419 31567 raft_consensus.cc:2258] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:17.164734 31568 raft_consensus.cc:2258] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:17.164997 31566 ts_tablet_manager.cc:1905] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:17.166321 31567 ts_tablet_manager.cc:1905] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:17.166330 30623 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:15:17.166958 30623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:15:17.170472 31568 ts_tablet_manager.cc:1905] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:17.171535 31566 ts_tablet_manager.cc:1918] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:17.171666 31566 log.cc:1196] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/64ecc9a319b34bd38da284f74652d006
I20260811 09:15:17.172034 31566 ts_tablet_manager.cc:1939] T 64ecc9a319b34bd38da284f74652d006 P a17f8ccef9f644f4a3d8a570baf6561d: Deleting consensus metadata
I20260811 09:15:17.173069 30593 catalog_manager.cc:4841] TS a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65:41163): tablet 64ecc9a319b34bd38da284f74652d006 (table test-table [id=02ba5d378072412aa34a268bd2494763]) successfully deleted
I20260811 09:15:17.173606 31567 ts_tablet_manager.cc:1918] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:17.173825 31567 log.cc:1196] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/64ecc9a319b34bd38da284f74652d006
I20260811 09:15:17.174015 30843 tablet_service.cc:1467] Processing CreateTablet for tablet ee42aac1f4554c48b9d5a6900c383da2 (DEFAULT_TABLE table=test-table [id=9f45bc2cca0b4955ae637240b0078c81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:17.174405 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee42aac1f4554c48b9d5a6900c383da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:17.174389 31567 ts_tablet_manager.cc:1939] T 64ecc9a319b34bd38da284f74652d006 P 2e3c5642bc4147eea8533ff8b62add6b: Deleting consensus metadata
I20260811 09:15:17.174512 31299 tablet_service.cc:1467] Processing CreateTablet for tablet ee42aac1f4554c48b9d5a6900c383da2 (DEFAULT_TABLE table=test-table [id=9f45bc2cca0b4955ae637240b0078c81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:17.175472 31060 tablet_service.cc:1467] Processing CreateTablet for tablet ee42aac1f4554c48b9d5a6900c383da2 (DEFAULT_TABLE table=test-table [id=9f45bc2cca0b4955ae637240b0078c81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:17.175897 31060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee42aac1f4554c48b9d5a6900c383da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:17.175951 31299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee42aac1f4554c48b9d5a6900c383da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:17.176960 31568 ts_tablet_manager.cc:1918] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:17.177110 31568 log.cc:1196] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/64ecc9a319b34bd38da284f74652d006
I20260811 09:15:17.177579 31568 ts_tablet_manager.cc:1939] T 64ecc9a319b34bd38da284f74652d006 P 2f560081c29641c88dad14da789187a5: Deleting consensus metadata
I20260811 09:15:17.178802 30590 catalog_manager.cc:4841] TS 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947): tablet 64ecc9a319b34bd38da284f74652d006 (table test-table [id=02ba5d378072412aa34a268bd2494763]) successfully deleted
W20260811 09:15:17.179461 30590 catalog_manager.cc:4808] TS 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947): delete failed for tablet 64ecc9a319b34bd38da284f74652d006 because the tablet was not found. No further retry: Not found: Tablet not found: 64ecc9a319b34bd38da284f74652d006
I20260811 09:15:17.180568 31592 tablet_bootstrap.cc:492] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d: Bootstrap starting.
I20260811 09:15:17.181958 31593 tablet_bootstrap.cc:492] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b: Bootstrap starting.
I20260811 09:15:17.182228 31592 tablet_bootstrap.cc:654] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:17.183794 31593 tablet_bootstrap.cc:654] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:17.184062 31604 tablet_bootstrap.cc:492] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5: Bootstrap starting.
I20260811 09:15:17.185709 31604 tablet_bootstrap.cc:654] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:17.188210 30595 catalog_manager.cc:4841] TS 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683): tablet 64ecc9a319b34bd38da284f74652d006 (table test-table [id=02ba5d378072412aa34a268bd2494763]) successfully deleted
I20260811 09:15:17.191418 31604 tablet_bootstrap.cc:492] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5: No bootstrap required, opened a new log
I20260811 09:15:17.191584 31604 ts_tablet_manager.cc:1397] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260811 09:15:17.192013 31593 tablet_bootstrap.cc:492] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b: No bootstrap required, opened a new log
I20260811 09:15:17.192170 31593 ts_tablet_manager.cc:1397] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260811 09:15:17.192373 31592 tablet_bootstrap.cc:492] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d: No bootstrap required, opened a new log
I20260811 09:15:17.192346 31604 raft_consensus.cc:348] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.192590 31592 ts_tablet_manager.cc:1397] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260811 09:15:17.192649 31604 raft_consensus.cc:374] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:17.192798 31604 raft_consensus.cc:729] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f560081c29641c88dad14da789187a5, State: Initialized, Role: FOLLOWER
I20260811 09:15:17.193017 31593 raft_consensus.cc:348] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.193145 31604 consensus_queue.cc:260] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.193285 31593 raft_consensus.cc:374] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:17.193359 31593 raft_consensus.cc:729] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e3c5642bc4147eea8533ff8b62add6b, State: Initialized, Role: FOLLOWER
I20260811 09:15:17.193377 31592 raft_consensus.cc:348] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.193683 31604 ts_tablet_manager.cc:1428] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 09:15:17.193687 31592 raft_consensus.cc:374] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:17.193768 31592 raft_consensus.cc:729] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17f8ccef9f644f4a3d8a570baf6561d, State: Initialized, Role: FOLLOWER
I20260811 09:15:17.194038 31592 consensus_queue.cc:260] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.194185 31593 consensus_queue.cc:260] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.195215 31592 ts_tablet_manager.cc:1428] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 09:15:17.196621 31593 ts_tablet_manager.cc:1428] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260811 09:15:17.286551 31552 raft_consensus.cc:482] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:15:17.286778 31552 raft_consensus.cc:504] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.287599 31552 leader_election.cc:288] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947), 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683)
I20260811 09:15:17.288213 31339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee42aac1f4554c48b9d5a6900c383da2" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3c5642bc4147eea8533ff8b62add6b" is_pre_election: true
I20260811 09:15:17.288206 31103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee42aac1f4554c48b9d5a6900c383da2" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f560081c29641c88dad14da789187a5" is_pre_election: true
I20260811 09:15:17.288509 31103 raft_consensus.cc:2454] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 0.
I20260811 09:15:17.288524 31339 raft_consensus.cc:2454] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 0.
I20260811 09:15:17.289021 30764 leader_election.cc:302] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f560081c29641c88dad14da789187a5, a17f8ccef9f644f4a3d8a570baf6561d; no voters:
I20260811 09:15:17.289361 31552 raft_consensus.cc:2781] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:15:17.289492 31552 raft_consensus.cc:482] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:15:17.289613 31552 raft_consensus.cc:3037] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:17.292614 31552 raft_consensus.cc:504] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.293303 31552 leader_election.cc:288] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 election: Requested vote from peers 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947), 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683)
I20260811 09:15:17.293815 31339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee42aac1f4554c48b9d5a6900c383da2" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e3c5642bc4147eea8533ff8b62add6b"
I20260811 09:15:17.293807 31103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee42aac1f4554c48b9d5a6900c383da2" candidate_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f560081c29641c88dad14da789187a5"
I20260811 09:15:17.294008 31339 raft_consensus.cc:3037] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:17.294042 31103 raft_consensus.cc:3037] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:17.296056 31339 raft_consensus.cc:2454] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 1.
I20260811 09:15:17.296452 30766 leader_election.cc:302] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e3c5642bc4147eea8533ff8b62add6b, a17f8ccef9f644f4a3d8a570baf6561d; no voters:
I20260811 09:15:17.296715 31552 raft_consensus.cc:2781] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:17.297122 31552 raft_consensus.cc:686] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 LEADER]: Becoming Leader. State: Replica: a17f8ccef9f644f4a3d8a570baf6561d, State: Running, Role: LEADER
I20260811 09:15:17.297128 31103 raft_consensus.cc:2454] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a17f8ccef9f644f4a3d8a570baf6561d in term 1.
I20260811 09:15:17.297405 31552 consensus_queue.cc:237] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } }
I20260811 09:15:17.300277 30617 catalog_manager.cc:5495] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d reported cstate change: term changed from 0 to 1, leader changed from <none> to a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65). New cstate: current_term: 1 leader_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a17f8ccef9f644f4a3d8a570baf6561d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 } health_report { overall_health: UNKNOWN } } }
I20260811 09:15:17.328814 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:17.323708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58680 (local address 127.17.16.65:41163)
0811 09:15:17.324467 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:17.324471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:17.324485 (+ 14us) server_negotiation.cc:407] Connection header received
0811 09:15:17.324534 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:17.324537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:17.324604 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:17.324698 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:15:17.325149 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:17.325887 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:17.327184 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:17.327576 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:17.327692 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:17.328108 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:15:17.328212 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:15:17.328284 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:17.328448 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 09:15:17.328525 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":551,"threads_started":1}
I20260811 09:15:17.337800 31339 raft_consensus.cc:1261] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Refusing update from remote peer a17f8ccef9f644f4a3d8a570baf6561d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:17.338239 31103 raft_consensus.cc:1261] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Refusing update from remote peer a17f8ccef9f644f4a3d8a570baf6561d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:17.338551 31552 consensus_queue.cc:1035] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e3c5642bc4147eea8533ff8b62add6b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 35683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:17.339542 31552 consensus_queue.cc:1035] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f560081c29641c88dad14da789187a5" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:17.394706 30617 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:17.394934 30617 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:17.397871 30617 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6337b7c373864632bbaecd34168e1ce4: Sending DeleteTablet for 3 replicas of tablet ee42aac1f4554c48b9d5a6900c383da2
I20260811 09:15:17.398554 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet ee42aac1f4554c48b9d5a6900c383da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:17 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:58668
I20260811 09:15:17.398707 31299 tablet_service.cc:1514] Processing DeleteTablet for tablet ee42aac1f4554c48b9d5a6900c383da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:17 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:40740
I20260811 09:15:17.398699 31060 tablet_service.cc:1514] Processing DeleteTablet for tablet ee42aac1f4554c48b9d5a6900c383da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:17 UTC) from {username='oryx', principal='oryx/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:53794
I20260811 09:15:17.398852 31566 tablet_replica.cc:330] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d: stopping tablet replica
I20260811 09:15:17.398965 31567 tablet_replica.cc:330] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b: stopping tablet replica
I20260811 09:15:17.399008 31568 tablet_replica.cc:330] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5: stopping tablet replica
I20260811 09:15:17.399021 31566 raft_consensus.cc:2229] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:15:17.399113 31567 raft_consensus.cc:2229] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:17.399187 31568 raft_consensus.cc:2229] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:17.399293 31567 raft_consensus.cc:2258] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:17.399361 31566 raft_consensus.cc:2258] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:17.399381 31568 raft_consensus.cc:2258] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:17.400569 31568 ts_tablet_manager.cc:1905] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:17.400593 31567 ts_tablet_manager.cc:1905] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:17.400846 31566 ts_tablet_manager.cc:1905] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:17.407025 31567 ts_tablet_manager.cc:1918] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:15:17.407025 31566 ts_tablet_manager.cc:1918] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:15:17.407162 31566 log.cc:1196] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/ee42aac1f4554c48b9d5a6900c383da2
I20260811 09:15:17.407550 31568 ts_tablet_manager.cc:1918] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:15:17.407605 31566 ts_tablet_manager.cc:1939] T ee42aac1f4554c48b9d5a6900c383da2 P a17f8ccef9f644f4a3d8a570baf6561d: Deleting consensus metadata
I20260811 09:15:17.407728 31568 log.cc:1196] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/ee42aac1f4554c48b9d5a6900c383da2
I20260811 09:15:17.408231 31568 ts_tablet_manager.cc:1939] T ee42aac1f4554c48b9d5a6900c383da2 P 2f560081c29641c88dad14da789187a5: Deleting consensus metadata
I20260811 09:15:17.408641 30593 catalog_manager.cc:4841] TS a17f8ccef9f644f4a3d8a570baf6561d (127.17.16.65:41163): tablet ee42aac1f4554c48b9d5a6900c383da2 (table test-table [id=9f45bc2cca0b4955ae637240b0078c81]) successfully deleted
I20260811 09:15:17.408846 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 30734
I20260811 09:15:17.409493 30590 catalog_manager.cc:4841] TS 2f560081c29641c88dad14da789187a5 (127.17.16.66:33947): tablet ee42aac1f4554c48b9d5a6900c383da2 (table test-table [id=9f45bc2cca0b4955ae637240b0078c81]) successfully deleted
I20260811 09:15:17.411495 31567 log.cc:1196] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/ee42aac1f4554c48b9d5a6900c383da2
I20260811 09:15:17.411871 31567 ts_tablet_manager.cc:1939] T ee42aac1f4554c48b9d5a6900c383da2 P 2e3c5642bc4147eea8533ff8b62add6b: Deleting consensus metadata
I20260811 09:15:17.412647 30595 catalog_manager.cc:4841] TS 2e3c5642bc4147eea8533ff8b62add6b (127.17.16.67:35683): tablet ee42aac1f4554c48b9d5a6900c383da2 (table test-table [id=9f45bc2cca0b4955ae637240b0078c81]) successfully deleted
Aug 11 09:15:17 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.65@KRBTEST.COM
I20260811 09:15:17.426184 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:17.410008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.17.16.65:41163)
0811 09:15:17.410198 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:17.410205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:17.410331 (+ 126us) server_negotiation.cc:407] Connection header received
0811 09:15:17.410631 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:17.410637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:17.410715 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:17.410791 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:17.412542 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:17.413397 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:17.415093 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:17.415452 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:17.415536 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:17.420321 (+ 4785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:17.420338 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:17.420355 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:17.420386 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:17.423354 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:17.424165 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:17.424172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:17.424174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:17.424310 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:17.424694 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:17.424697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:17.424699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:17.425160 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:17.425404 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:17.425734 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 09:15:17.425912 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 09:15:17.426801 30946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:17.621913 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 30734
I20260811 09:15:17.660580 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 30960
Aug 11 09:15:17 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.66@KRBTEST.COM
I20260811 09:15:17.674610 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:17.661382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53836 (local address 127.17.16.66:33947)
0811 09:15:17.661836 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:17.661841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:17.661857 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:15:17.661912 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:17.661916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:17.661987 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:17.662076 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:17.663244 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:17.663948 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:17.665163 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:17.665570 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:17.665670 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:17.669659 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:17.669672 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:17.669688 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:17.669722 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:17.672277 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:17.672868 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:17.672874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:17.672877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:17.673123 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:17.673402 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:17.673405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:17.673407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:17.673738 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:17.673950 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:17.674159 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 09:15:17.674322 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":241,"threads_started":1}
I20260811 09:15:17.675223 31174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:17.817065 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 30960
I20260811 09:15:17.817205 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:17.816213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:41163 (local address 127.17.16.66:32895)
0811 09:15:17.816826 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:17.816981 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:41163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":361,"threads_started":1}
I20260811 09:15:17.849609 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 31191
Aug 11 09:15:17 dist-test-slave-ncp9 krb5kdc[30539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.16.67@KRBTEST.COM
I20260811 09:15:17.863909 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:17.850392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.17.16.67:35683)
0811 09:15:17.850822 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:17.850826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:17.850842 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:15:17.850899 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:17.850903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:17.850964 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:17.851054 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:17.852378 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:17.853217 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:17.854374 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:17.854778 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:17.854861 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:17.858878 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:17.858894 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:17.858911 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:17.858947 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:17.861576 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:17.862187 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:17.862206 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:17.862208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:17.862336 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:17.862622 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:17.862625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:17.862627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:17.862975 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:17.863209 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:17.863400 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 09:15:17.863581 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":231,"threads_started":1}
I20260811 09:15:17.864602 31404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:17.959273 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 31191
I20260811 09:15:17.982405 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 30558
I20260811 09:15:17.983543 30681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:18.097882 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 30558
2026-08-11T09:15:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7202 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:15:18 dist-test-slave-ncp9 krb5kdc[31649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:15:18 dist-test-slave-ncp9 krb5kdc[31649](info): set up 2 sockets
Aug 11 09:15:18 dist-test-slave-ncp9 krb5kdc[31649](info): commencing operation
krb5kdc: starting...
W20260811 09:15:19.327442 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:15:19 dist-test-slave-ncp9 krb5kdc[31649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:15:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:15:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:15:19.577466 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:43051
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:37799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:43051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:19.905503 31672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:19.905773 31672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:19.905860 31672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:19.913025 31672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:15:19.913127 31672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:19.913180 31672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:19.913236 31672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:15:19.913285 31672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:15:19.922876 31672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:43051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:43051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:19.924045 31672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:19.925272 31672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:19.936596 31682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:19.936626 31680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:19.936654 31686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:19.937690 31672 server_base.cc:1034] running on GCE node
I20260811 09:15:19.938364 31672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:19.939699 31672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:19.940949 31672 hybrid_clock.cc:648] HybridClock initialized: now 1786439719940869 us; error 28 us; skew 500 ppm
I20260811 09:15:19.941291 31672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T09:15:20Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1904 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:15:20 dist-test-slave-ncp9 krb5kdc[31697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:15:20 dist-test-slave-ncp9 krb5kdc[31697](info): set up 2 sockets
Aug 11 09:15:20 dist-test-slave-ncp9 krb5kdc[31697](info): commencing operation
krb5kdc: starting...
W20260811 09:15:21.240172 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:15:21 dist-test-slave-ncp9 krb5kdc[31697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:15:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:15:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:15:21.492538 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46083
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:46083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:21.813187 31725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:21.813450 31725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:21.813547 31725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:21.820537 31725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:15:21.820650 31725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:21.820703 31725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:21.820758 31725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:15:21.820801 31725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:15:21.830425 31725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:46083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:21.831642 31725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:21.832926 31725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:21.844049 31732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:21.844007 31733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:21.844084 31735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:21.844837 31725 server_base.cc:1034] running on GCE node
I20260811 09:15:21.845578 31725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:21.846866 31725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:21.848059 31725 hybrid_clock.cc:648] HybridClock initialized: now 1786439721848016 us; error 31 us; skew 500 ppm
I20260811 09:15:21.848368 31725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:21 dist-test-slave-ncp9 krb5kdc[31697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439721, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:21.855075 31725 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:15:21.856410 31725 webserver.cc:460] Webserver started at http://127.17.16.126:42065/ using document root <none> and password file <none>
I20260811 09:15:21.857107 31725 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:21.857225 31725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:21.857515 31725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:21.860339 31725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "adfe7039edff4a54adbb22b5f7ba51c9"
format_stamp: "Formatted at 2026-08-11 09:15:21 on dist-test-slave-ncp9"
server_key: "6e89bb8712b03dd752a7e77b6ceb1bd3"
server_key_iv: "1145fa25b8e89c35e17bb434f5feec75"
server_key_version: "encryptionkey@0"
I20260811 09:15:21.861104 31725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "adfe7039edff4a54adbb22b5f7ba51c9"
format_stamp: "Formatted at 2026-08-11 09:15:21 on dist-test-slave-ncp9"
server_key: "6e89bb8712b03dd752a7e77b6ceb1bd3"
server_key_iv: "1145fa25b8e89c35e17bb434f5feec75"
server_key_version: "encryptionkey@0"
I20260811 09:15:21.865921 31725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:15:21.869082 31745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:21.870256 31725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 09:15:21.870406 31725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "adfe7039edff4a54adbb22b5f7ba51c9"
format_stamp: "Formatted at 2026-08-11 09:15:21 on dist-test-slave-ncp9"
server_key: "6e89bb8712b03dd752a7e77b6ceb1bd3"
server_key_iv: "1145fa25b8e89c35e17bb434f5feec75"
server_key_version: "encryptionkey@0"
I20260811 09:15:21.870580 31725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:21.896011 31725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:21.900084 31725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:21.900348 31725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:21.917196 31725 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:46083
I20260811 09:15:21.917229 31830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:46083 every 8 connection(s)
I20260811 09:15:21.918459 31725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:15:21.921937 31831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:21.924316 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 31725
I20260811 09:15:21.924542 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:15:21.925357 17473 external_mini_cluster.cc:1428] Setting key 44a391ad389a17fd788dcd5146c131f9
I20260811 09:15:21.933959 31831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9: Bootstrap starting.
I20260811 09:15:21.937493 31831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:21.938756 31831 log.cc:824] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:15:21 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:15:21.941629 31831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9: No bootstrap required, opened a new log
I20260811 09:15:21.947882 31836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:21.927351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50008 (local address 127.17.16.126:46083)
0811 09:15:21.928072 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:21.928082 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:21.928124 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:15:21.929814 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:21.929836 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:21.930182 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:21.930510 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:21.932257 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:21.933595 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:21.935302 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:21.935988 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:21.936112 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:21.940778 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:21.940787 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:21.940801 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:21.940835 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:21.943804 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:21.944461 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:21.944467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:21.944473 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:21.944671 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:21.945177 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:21.945181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:21.945183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:21.945739 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:21.946036 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:21.946508 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 09:15:21.946907 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":322,"threads_started":1}
I20260811 09:15:21.949282 31831 raft_consensus.cc:348] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } }
I20260811 09:15:21.949698 31831 raft_consensus.cc:374] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:21.949790 31831 raft_consensus.cc:729] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfe7039edff4a54adbb22b5f7ba51c9, State: Initialized, Role: FOLLOWER
I20260811 09:15:21.950336 31831 consensus_queue.cc:260] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } }
I20260811 09:15:21.950605 31831 raft_consensus.cc:388] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:15:21.950754 31831 raft_consensus.cc:482] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:15:21.950886 31831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:21.953006 31831 raft_consensus.cc:504] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } }
I20260811 09:15:21.953438 31831 leader_election.cc:302] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adfe7039edff4a54adbb22b5f7ba51c9; no voters:
I20260811 09:15:21.953943 31831 leader_election.cc:288] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:15:21.954041 31839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:21.954661 31839 raft_consensus.cc:686] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 1 LEADER]: Becoming Leader. State: Replica: adfe7039edff4a54adbb22b5f7ba51c9, State: Running, Role: LEADER
I20260811 09:15:21.955138 31839 consensus_queue.cc:237] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } }
I20260811 09:15:21.956007 31831 sys_catalog.cc:564] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:15:21.958045 31842 sys_catalog.cc:455] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adfe7039edff4a54adbb22b5f7ba51c9. Latest consensus state: current_term: 1 leader_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } } }
I20260811 09:15:21.958016 31841 sys_catalog.cc:455] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } } }
I20260811 09:15:21.958485 31842 sys_catalog.cc:458] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:21.958487 31841 sys_catalog.cc:458] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:21.961184 31851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:15:21.965232 31851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:15:21.972823 31851 catalog_manager.cc:1347] Generated new cluster ID: 8022cc7c95b04942b5cd5d110c2aa9ad
I20260811 09:15:21.972956 31851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:15:21.984398 31851 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:15:21.986158 31851 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:15:21.999732 31851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9: Generated new TSK 0
I20260811 09:15:22.000725 31851 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:15:22.062819 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46083
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:22.391601 31872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:22.391862 31872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:22.391979 31872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:22.399045 31872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:22.399147 31872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:22.399294 31872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:15:22.408838 31872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:22.410077 31872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:22.411362 31872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:22.423735 31879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:22.423772 31881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:22.423870 31878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:22.424623 31872 server_base.cc:1034] running on GCE node
I20260811 09:15:22.425199 31872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:22.426019 31872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:22.427213 31872 hybrid_clock.cc:648] HybridClock initialized: now 1786439722427175 us; error 28 us; skew 500 ppm
I20260811 09:15:22.427495 31872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:22 dist-test-slave-ncp9 krb5kdc[31697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439722, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:22.430437 31872 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:15:22.431770 31872 webserver.cc:460] Webserver started at http://127.17.16.65:41391/ using document root <none> and password file <none>
I20260811 09:15:22.432391 31872 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:22.432502 31872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:22.432801 31872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:22.435559 31872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "3666d200d9d54a43b5e0301610d080e7"
format_stamp: "Formatted at 2026-08-11 09:15:22 on dist-test-slave-ncp9"
server_key: "99bdfdc408eed9553271053b12dffe8e"
server_key_iv: "042f2b35191025356d8148accfec4581"
server_key_version: "encryptionkey@0"
I20260811 09:15:22.436326 31872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "3666d200d9d54a43b5e0301610d080e7"
format_stamp: "Formatted at 2026-08-11 09:15:22 on dist-test-slave-ncp9"
server_key: "99bdfdc408eed9553271053b12dffe8e"
server_key_iv: "042f2b35191025356d8148accfec4581"
server_key_version: "encryptionkey@0"
I20260811 09:15:22.441259 31872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:15:22.445144 31890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:22.446707 31872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:15:22.447921 31872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "3666d200d9d54a43b5e0301610d080e7"
format_stamp: "Formatted at 2026-08-11 09:15:22 on dist-test-slave-ncp9"
server_key: "99bdfdc408eed9553271053b12dffe8e"
server_key_iv: "042f2b35191025356d8148accfec4581"
server_key_version: "encryptionkey@0"
I20260811 09:15:22.448143 31872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:22.464241 31872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:22.469632 31872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:22.469910 31872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:22.470870 31872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:22.472424 31872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:22.472496 31872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:22.472591 31872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:22.472637 31872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:22.501153 31872 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:45279
I20260811 09:15:22.501189 32083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:45279 every 8 connection(s)
I20260811 09:15:22.502478 31872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:15:22.508371 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 31872
I20260811 09:15:22.508587 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:15:22.509233 17473 external_mini_cluster.cc:1428] Setting key b397d7ee22c4f37f185b2f1138f5d4a4
Aug 11 09:15:22 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439722, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:15:22.521220 32089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:22.504865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:43579 (local address 127.17.16.126:46083)
0811 09:15:22.505312 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:22.505316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:22.506168 (+ 852us) server_negotiation.cc:407] Connection header received
0811 09:15:22.507956 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:22.507962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:22.508046 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:22.508143 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:22.509733 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:22.510564 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:22.512161 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:22.512550 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:22.512666 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:22.515899 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:22.515904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:22.515908 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:22.515938 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:22.518333 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:22.518996 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:22.519001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:22.519003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:22.519121 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:22.519523 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:22.519526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:22.519528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:22.519790 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:22.519921 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:22.520603 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 09:15:22.520848 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":197,"threads_started":1}
I20260811 09:15:22.522087 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:22.505341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:43579)
0811 09:15:22.506008 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:22.506056 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:22.507691 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:22.508383 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:22.508392 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:22.508978 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:22.509562 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:22.509582 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:22.510738 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:22.510742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:22.512019 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:22.512037 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:22.512764 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:22.512772 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:22.513033 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:22.513641 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:22.513663 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:22.515699 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:22.518458 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:22.518467 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:22.518482 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:22.518877 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:22.519224 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:22.519229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:22.519232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:22.519411 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:22.519911 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:22.519919 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:22.520233 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 09:15:22.520465 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 09:15:22.520991 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":261,"threads_started":1}
I20260811 09:15:22.523537 32084 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46083
I20260811 09:15:22.523852 32084 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:22.524713 32084 heartbeater.cc:511] Master 127.17.16.126:46083 requested a full tablet report, sending...
I20260811 09:15:22.526998 31775 ts_manager.cc:194] Registered new tserver with Master: 3666d200d9d54a43b5e0301610d080e7 (127.17.16.65:45279)
I20260811 09:15:22.528880 31775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:43579
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:15:22.566073 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46083
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:22.886368 32095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:22.886633 32095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:22.886736 32095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:22.893618 32095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:22.893710 32095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:22.893823 32095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:15:22.902848 32095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:22.903923 32095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:22.905155 32095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:22.917214 32101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:22.917430 32100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:22.917655 32105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:22.918151 32095 server_base.cc:1034] running on GCE node
I20260811 09:15:22.918816 32095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:22.920012 32095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:22.921240 32095 hybrid_clock.cc:648] HybridClock initialized: now 1786439722921155 us; error 71 us; skew 500 ppm
I20260811 09:15:22.921586 32095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:22 dist-test-slave-ncp9 krb5kdc[31697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439722, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:22.925271 32095 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:15:22.926785 32095 webserver.cc:460] Webserver started at http://127.17.16.66:37397/ using document root <none> and password file <none>
I20260811 09:15:22.927532 32095 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:22.927657 32095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:22.927983 32095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:22.931723 32095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "14a623e1c2414955a747ed378eb07fd6"
format_stamp: "Formatted at 2026-08-11 09:15:22 on dist-test-slave-ncp9"
server_key: "9a446026a4df71e597b772f6fa836887"
server_key_iv: "ce9992bb5a029437c434ee2d02cd22f2"
server_key_version: "encryptionkey@0"
I20260811 09:15:22.932698 32095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "14a623e1c2414955a747ed378eb07fd6"
format_stamp: "Formatted at 2026-08-11 09:15:22 on dist-test-slave-ncp9"
server_key: "9a446026a4df71e597b772f6fa836887"
server_key_iv: "ce9992bb5a029437c434ee2d02cd22f2"
server_key_version: "encryptionkey@0"
I20260811 09:15:22.938457 32095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:15:22.941671 32116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:22.942867 32095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:22.942996 32095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "14a623e1c2414955a747ed378eb07fd6"
format_stamp: "Formatted at 2026-08-11 09:15:22 on dist-test-slave-ncp9"
server_key: "9a446026a4df71e597b772f6fa836887"
server_key_iv: "ce9992bb5a029437c434ee2d02cd22f2"
server_key_version: "encryptionkey@0"
I20260811 09:15:22.943166 32095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:22.960095 32095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:22.963852 32095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:22.964094 32095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:22.964993 32095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:22.966316 32095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:22.966408 32095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:22.966521 32095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:22.966566 32095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:22.995571 32095 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:37983
I20260811 09:15:22.995607 32283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:37983 every 8 connection(s)
I20260811 09:15:22.996763 32095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:15:22.998976 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 32095
I20260811 09:15:22.999203 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:15:22.999625 17473 external_mini_cluster.cc:1428] Setting key b06e4a0c8ef55bcfbd9d58dcd0a942ad
Aug 11 09:15:23 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439722, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:15:23.015908 32089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:22.999235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:40341 (local address 127.17.16.126:46083)
0811 09:15:22.999465 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:22.999471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:23.000710 (+ 1239us) server_negotiation.cc:407] Connection header received
0811 09:15:23.002544 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:23.002550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:23.002628 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:23.002738 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:23.004123 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.004829 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.006344 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.006763 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.006869 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:23.010377 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:23.010382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:23.010386 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:23.010418 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:23.013113 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:23.013765 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:23.013772 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:23.013776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:23.013934 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:23.014299 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:23.014303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:23.014306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:23.014629 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:23.014772 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:23.015372 (+ 600us) server_negotiation.cc:299] Negotiation successful
0811 09:15:23.015604 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 09:15:23.016589 32289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:22.999760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:40341)
0811 09:15:23.000545 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:23.000600 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:23.002288 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:23.002911 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:23.002919 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:23.003398 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:23.003944 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:23.003962 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.005010 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.005015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:23.006197 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:23.006210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.006975 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.006983 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:23.007190 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:23.007799 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:23.007824 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:23.010163 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:23.013237 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:23.013243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:23.013255 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:23.013649 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:23.014029 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:23.014032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:23.014035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:23.014203 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:23.014730 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:23.014736 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:23.015021 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 09:15:23.015244 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 09:15:23.015583 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":316,"threads_started":1}
I20260811 09:15:23.018075 32287 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46083
I20260811 09:15:23.018366 32287 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:23.019130 32287 heartbeater.cc:511] Master 127.17.16.126:46083 requested a full tablet report, sending...
I20260811 09:15:23.020596 31775 ts_manager.cc:194] Registered new tserver with Master: 14a623e1c2414955a747ed378eb07fd6 (127.17.16.66:37983)
I20260811 09:15:23.021414 31775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:40341
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:15:23.057324 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46083
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:23.377501 32292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:23.377777 32292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:23.377916 32292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:23.384860 32292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:23.384995 32292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:23.385124 32292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:15:23.394502 32292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:23.395684 32292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:23.396944 32292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:23.409505 32299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:23.409582 32301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:23.409602 32304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:23.410285 32292 server_base.cc:1034] running on GCE node
I20260811 09:15:23.410792 32292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:23.411599 32292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:23.412786 32292 hybrid_clock.cc:648] HybridClock initialized: now 1786439723412743 us; error 35 us; skew 500 ppm
I20260811 09:15:23.413085 32292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:23 dist-test-slave-ncp9 krb5kdc[31697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439723, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:23.416025 32292 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:15:23.417387 32292 webserver.cc:460] Webserver started at http://127.17.16.67:35581/ using document root <none> and password file <none>
I20260811 09:15:23.418010 32292 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:23.418105 32292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:23.418344 32292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:23.421054 32292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "c039ff13a7f1475f859e51e4b2b652f7"
format_stamp: "Formatted at 2026-08-11 09:15:23 on dist-test-slave-ncp9"
server_key: "20c9cd7c0404034a6a910e69ad01148b"
server_key_iv: "48445a8aa50cc72176ed74837e09519b"
server_key_version: "encryptionkey@0"
I20260811 09:15:23.421793 32292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "c039ff13a7f1475f859e51e4b2b652f7"
format_stamp: "Formatted at 2026-08-11 09:15:23 on dist-test-slave-ncp9"
server_key: "20c9cd7c0404034a6a910e69ad01148b"
server_key_iv: "48445a8aa50cc72176ed74837e09519b"
server_key_version: "encryptionkey@0"
I20260811 09:15:23.426450 32292 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:15:23.429533 32322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:23.430768 32292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:23.430907 32292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "c039ff13a7f1475f859e51e4b2b652f7"
format_stamp: "Formatted at 2026-08-11 09:15:23 on dist-test-slave-ncp9"
server_key: "20c9cd7c0404034a6a910e69ad01148b"
server_key_iv: "48445a8aa50cc72176ed74837e09519b"
server_key_version: "encryptionkey@0"
I20260811 09:15:23.431074 32292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:23.450096 32292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:23.454316 32292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:23.454581 32292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:23.455514 32292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:23.456852 32292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:23.456956 32292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:23.457077 32292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:23.457121 32292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:23.485502 32292 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:44385
I20260811 09:15:23.485543 32508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:44385 every 8 connection(s)
I20260811 09:15:23.486714 32292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:15:23.490906 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 32292
I20260811 09:15:23.491097 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:15:23.491492 17473 external_mini_cluster.cc:1428] Setting key 0ae3e7562e2e296040bb2443872b3ea1
Aug 11 09:15:23 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439723, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:15:23.506011 32089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:23.489099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:60489 (local address 127.17.16.126:46083)
0811 09:15:23.489306 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:23.489313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:23.490470 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 09:15:23.492381 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:23.492387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:23.492480 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:23.492567 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:23.494461 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.495221 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.496716 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.497168 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.497277 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:23.500558 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:23.500563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:23.500567 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:23.500601 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:23.503061 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:23.503735 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:23.503741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:23.503744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:23.503870 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:23.504281 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:23.504286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:23.504289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:23.504550 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:23.504694 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:23.505440 (+ 746us) server_negotiation.cc:299] Negotiation successful
0811 09:15:23.505674 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:15:23.506973 32512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:23.489604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:60489)
0811 09:15:23.490303 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:23.490356 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:23.492110 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:23.492840 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:23.492850 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:23.493503 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:23.494304 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:23.494325 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.495374 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.495379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:23.496578 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:23.496591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.497388 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.497395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:23.497629 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:23.498212 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:23.498234 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:23.500364 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:23.503199 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:23.503208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:23.503223 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:23.503620 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:23.503974 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:23.503979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:23.503983 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:23.504173 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:23.504676 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:23.504683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:23.505041 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 09:15:23.505347 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 09:15:23.505833 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":258,"threads_started":1}
I20260811 09:15:23.508082 32509 heartbeater.cc:347] Connected to a master server at 127.17.16.126:46083
I20260811 09:15:23.508368 32509 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:23.509091 32509 heartbeater.cc:511] Master 127.17.16.126:46083 requested a full tablet report, sending...
I20260811 09:15:23.510356 31775 ts_manager.cc:194] Registered new tserver with Master: c039ff13a7f1475f859e51e4b2b652f7 (127.17.16.67:44385)
I20260811 09:15:23.511106 31775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:60489
I20260811 09:15:23.518537 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:15:23.532285 32084 heartbeater.cc:503] Master 127.17.16.126:46083 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab.
Entry for principal oryx/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab.
WARNING: no policy specified for oryx/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab.
Entry for principal oryx/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab.
WARNING: no policy specified for oryx/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab.
Entry for principal oryx/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab.
I20260811 09:15:23.659207 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 31872
Aug 11 09:15:23 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:15:23.673967 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:23.660302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.17.16.65:45279)
0811 09:15:23.660945 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:23.660954 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:23.660983 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:15:23.661072 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:23.661081 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:23.661339 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:23.661468 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:23.662750 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.663734 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.664872 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.665435 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.665538 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:23.669163 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:23.669174 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:23.669187 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:23.669224 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:23.671652 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:23.672243 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:23.672249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:23.672251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:23.672371 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:23.672627 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:23.672630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:23.672632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:23.673020 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:23.673240 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:23.673452 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 09:15:23.673671 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":317,"threads_started":1}
I20260811 09:15:23.675283 32077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:23.757208 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 31872
I20260811 09:15:23.779203 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 32095
Aug 11 09:15:23 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:15:23.793972 32525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:23.780024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54906 (local address 127.17.16.66:37983)
0811 09:15:23.780609 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:23.780616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:23.780653 (+ 37us) server_negotiation.cc:407] Connection header received
0811 09:15:23.780726 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:23.780732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:23.780985 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:23.781117 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:23.782285 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.783268 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.784432 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.785125 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.785219 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:23.789131 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:23.789143 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:23.789158 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:23.789191 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:23.791720 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:23.792262 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:23.792267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:23.792270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:23.792394 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:23.792670 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:23.792672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:23.792674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:23.793039 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:23.793264 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:23.793478 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 09:15:23.793679 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":316,"threads_started":1}
I20260811 09:15:23.795220 32279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:23.875718 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 32095
I20260811 09:15:23.894181 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 32292
Aug 11 09:15:23 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:15:23.910104 32531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:23.895150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.17.16.67:44385)
0811 09:15:23.895769 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:23.895776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:23.895805 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:15:23.895869 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:23.895875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:23.896090 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:23.896237 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:23.897644 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.898684 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.899934 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:23.900506 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:23.900619 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:23.905035 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:23.905048 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:23.905063 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:23.905098 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:23.907795 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:23.908374 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:23.908380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:23.908382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:23.908507 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:23.908789 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:23.908792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:23.908794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:23.909169 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:23.909403 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:23.909601 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 09:15:23.909793 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":300,"threads_started":1}
I20260811 09:15:23.911365 32495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:23.992569 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 32292
I20260811 09:15:24.015251 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 31725
I20260811 09:15:24.016407 31824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:24.117563 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 31725
I20260811 09:15:24.144542 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46083
--webserver_interface=127.17.16.126
--webserver_port=42065
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:46083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:24.466828 32534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:24.467111 32534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:24.467237 32534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:24.474150 32534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:15:24.474251 32534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:24.474332 32534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:24.474411 32534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:15:24.474466 32534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:15:24.483857 32534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:46083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=42065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:24.485169 32534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:24.486400 32534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:24.497469 32540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:24.497524 32544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:24.497731 32539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:24.498243 32534 server_base.cc:1034] running on GCE node
I20260811 09:15:24.499009 32534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:24.500315 32534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:24.501518 32534 hybrid_clock.cc:648] HybridClock initialized: now 1786439724501473 us; error 32 us; skew 500 ppm
I20260811 09:15:24.501855 32534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:24 dist-test-slave-ncp9 krb5kdc[31697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439724, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:24.504810 32534 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:15:24.506270 32534 webserver.cc:460] Webserver started at http://127.17.16.126:42065/ using document root <none> and password file <none>
I20260811 09:15:24.506924 32534 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:24.507063 32534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:24.512245 32534 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:15:24.514932 32557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:24.516137 32534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:15:24.516292 32534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "adfe7039edff4a54adbb22b5f7ba51c9"
format_stamp: "Formatted at 2026-08-11 09:15:21 on dist-test-slave-ncp9"
server_key: "6e89bb8712b03dd752a7e77b6ceb1bd3"
server_key_iv: "1145fa25b8e89c35e17bb434f5feec75"
server_key_version: "encryptionkey@0"
I20260811 09:15:24.517290 32534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:24.539767 32534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:24.543987 32534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:24.544241 32534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:24.561094 32534 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:46083
I20260811 09:15:24.561101 32641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:46083 every 8 connection(s)
I20260811 09:15:24.562523 32534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:15:24.566819 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 32534
I20260811 09:15:24.567776 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:45279
--local_ip_for_outbound_sockets=127.17.16.65
--tserver_master_addrs=127.17.16.126:46083
--webserver_port=41391
--webserver_interface=127.17.16.65
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.16.65
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:15:24.570061 32642 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:15:24.574158 32642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9: Bootstrap starting.
I20260811 09:15:24.593849 32642 log.cc:824] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:24.605033 32642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:15:24.605674 32642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9: Bootstrap complete.
I20260811 09:15:24.614701 32642 raft_consensus.cc:348] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } }
I20260811 09:15:24.615527 32642 raft_consensus.cc:729] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfe7039edff4a54adbb22b5f7ba51c9, State: Initialized, Role: FOLLOWER
I20260811 09:15:24.616086 32642 consensus_queue.cc:260] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } }
I20260811 09:15:24.616317 32642 raft_consensus.cc:388] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:15:24.616426 32642 raft_consensus.cc:482] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:15:24.616580 32642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:15:24.620330 32642 raft_consensus.cc:504] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } }
I20260811 09:15:24.620810 32642 leader_election.cc:302] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adfe7039edff4a54adbb22b5f7ba51c9; no voters:
I20260811 09:15:24.621454 32642 leader_election.cc:288] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:15:24.621589 32648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:15:24.623541 32642 sys_catalog.cc:564] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:15:24.628535 32648 raft_consensus.cc:686] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [term 2 LEADER]: Becoming Leader. State: Replica: adfe7039edff4a54adbb22b5f7ba51c9, State: Running, Role: LEADER
I20260811 09:15:24.629325 32648 consensus_queue.cc:237] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } }
I20260811 09:15:24.632136 32649 sys_catalog.cc:455] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } } }
I20260811 09:15:24.632504 32649 sys_catalog.cc:458] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:24.633070 32650 sys_catalog.cc:455] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adfe7039edff4a54adbb22b5f7ba51c9. Latest consensus state: current_term: 2 leader_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfe7039edff4a54adbb22b5f7ba51c9" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46083 } } }
I20260811 09:15:24.633371 32650 sys_catalog.cc:458] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:24.639948 32661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:15:24.645426 32661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:15:24.646845 32661 catalog_manager.cc:1259] Loaded cluster ID: 8022cc7c95b04942b5cd5d110c2aa9ad
I20260811 09:15:24.646999 32661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:15:24.650975 32661 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:15:24.652369 32661 catalog_manager.cc:5879] T 00000000000000000000000000000000 P adfe7039edff4a54adbb22b5f7ba51c9: Loaded TSK: 0
I20260811 09:15:24.653891 32661 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:15:24.921729 32646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:24.922065 32646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:24.922284 32646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:24.929728 32646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:24.929852 32646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:24.930011 32646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:15:24.939888 32646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.65.keytab
--principal=oryx/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:45279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=41391
--tserver_master_addrs=127.17.16.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:24.941290 32646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:24.942633 32646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:24.957197 32682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:24.957643 32679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:24.959503 32646 server_base.cc:1034] running on GCE node
W20260811 09:15:24.959712 32684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:24.960342 32646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:24.961320 32646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:24.962517 32646 hybrid_clock.cc:648] HybridClock initialized: now 1786439724962457 us; error 57 us; skew 500 ppm
I20260811 09:15:24.962813 32646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:24 dist-test-slave-ncp9 krb5kdc[31697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439724, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:24.965819 32646 init.cc:377] Logged in from keytab as oryx/127.17.16.65@KRBTEST.COM (short username oryx)
I20260811 09:15:24.967273 32646 webserver.cc:460] Webserver started at http://127.17.16.65:41391/ using document root <none> and password file <none>
I20260811 09:15:24.967906 32646 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:24.968035 32646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:24.973294 32646 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:15:24.975972 32692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:24.977427 32646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:24.977574 32646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "3666d200d9d54a43b5e0301610d080e7"
format_stamp: "Formatted at 2026-08-11 09:15:22 on dist-test-slave-ncp9"
server_key: "99bdfdc408eed9553271053b12dffe8e"
server_key_iv: "042f2b35191025356d8148accfec4581"
server_key_version: "encryptionkey@0"
I20260811 09:15:24.978494 32646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:25.002939 32646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:25.006963 32646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:25.007218 32646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:25.008162 32646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:25.009512 32646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:25.009613 32646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:25.009714 32646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:25.009784 32646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:25.040972 32646 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:45279
I20260811 09:15:25.041034 402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:45279 every 8 connection(s)
I20260811 09:15:25.042514 32646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:15:25.045972 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 32646
I20260811 09:15:25.046645 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:37983
--local_ip_for_outbound_sockets=127.17.16.66
--tserver_master_addrs=127.17.16.126:46083
--webserver_port=37397
--webserver_interface=127.17.16.66
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.16.66
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:15:25 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:25 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:15:25.387297 411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:25.387576 411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:25.387719 411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:25.394770 411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:25.394893 411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:25.395038 411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:15:25.404392 411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.66.keytab
--principal=oryx/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:37983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=37397
--tserver_master_addrs=127.17.16.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:25.405694 411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:25.406999 411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:25.420745 423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:25.420776 419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:25.421409 418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:25.422413 411 server_base.cc:1034] running on GCE node
I20260811 09:15:25.422998 411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:25.423935 411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:25.425143 411 hybrid_clock.cc:648] HybridClock initialized: now 1786439725425087 us; error 48 us; skew 500 ppm
I20260811 09:15:25.425427 411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:25 dist-test-slave-ncp9 krb5kdc[31697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439725, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:25.428412 411 init.cc:377] Logged in from keytab as oryx/127.17.16.66@KRBTEST.COM (short username oryx)
I20260811 09:15:25.429930 411 webserver.cc:460] Webserver started at http://127.17.16.66:37397/ using document root <none> and password file <none>
I20260811 09:15:25.430568 411 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:25.430691 411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:25.435823 411 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 09:15:25.438625 433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:25.440100 411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:15:25.440258 411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "14a623e1c2414955a747ed378eb07fd6"
format_stamp: "Formatted at 2026-08-11 09:15:22 on dist-test-slave-ncp9"
server_key: "9a446026a4df71e597b772f6fa836887"
server_key_iv: "ce9992bb5a029437c434ee2d02cd22f2"
server_key_version: "encryptionkey@0"
I20260811 09:15:25.441229 411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:25.473683 411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:25.477797 411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:25.478042 411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:25.478995 411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:25.480727 411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:25.480849 411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:25.480984 411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:25.481065 411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:25.522069 411 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:37983
I20260811 09:15:25.522137 612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:37983 every 8 connection(s)
I20260811 09:15:25.523718 411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:15:25.528265 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 411
I20260811 09:15:25.529084 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:44385
--local_ip_for_outbound_sockets=127.17.16.67
--tserver_master_addrs=127.17.16.126:46083
--webserver_port=35581
--webserver_interface=127.17.16.67
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.16.67
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:15:25 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:25 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:15:26.040789 617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:26.041260 617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:26.041435 617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:26.051470 617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:26.051594 617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:26.051779 617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:15:26.062112 617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/krb5kdc/oryx_127.17.16.67.keytab
--principal=oryx/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:44385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=35581
--tserver_master_addrs=127.17.16.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:26.063443 617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:26.064795 617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:26.085811 628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:26.087049 627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:26.091058 631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:26.091773 617 server_base.cc:1034] running on GCE node
I20260811 09:15:26.092571 617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:26.093776 617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:26.095011 617 hybrid_clock.cc:648] HybridClock initialized: now 1786439726094935 us; error 69 us; skew 500 ppm
I20260811 09:15:26.095410 617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439726, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:26.099905 617 init.cc:377] Logged in from keytab as oryx/127.17.16.67@KRBTEST.COM (short username oryx)
I20260811 09:15:26.101995 617 webserver.cc:460] Webserver started at http://127.17.16.67:35581/ using document root <none> and password file <none>
I20260811 09:15:26.103051 617 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:26.103231 617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:26.110158 617 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:15:26.113619 642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:26.115288 617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:15:26.115446 617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "c039ff13a7f1475f859e51e4b2b652f7"
format_stamp: "Formatted at 2026-08-11 09:15:23 on dist-test-slave-ncp9"
server_key: "20c9cd7c0404034a6a910e69ad01148b"
server_key_iv: "48445a8aa50cc72176ed74837e09519b"
server_key_version: "encryptionkey@0"
I20260811 09:15:26.116770 617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:26.141552 617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:26.147110 617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:26.147454 617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:26.148856 617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:26.150671 617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:26.150786 617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:26.150947 617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:26.151026 617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20260811 09:15:26.151216 409 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.094s user 0.008s sys 0.011s
I20260811 09:15:26.153391 409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:25.045613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:34793)
0811 09:15:25.046743 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:25.046802 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:25.048748 (+ 1946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:25.050322 (+ 1574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:25.050330 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:25.050864 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:25.051421 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:25.051441 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:25.053580 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:25.053586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:25.055263 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:25.055281 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:25.056529 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:25.056535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:25.056787 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:25.057532 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:25.057559 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:26.152162 (+1094603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":839,"thread_start_us":645,"threads_started":1}
W20260811 09:15:26.153739 407 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:25.045550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:34793 (local address 127.17.16.126:46083)
0811 09:15:25.046311 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:25.046327 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:25.046957 (+ 630us) server_negotiation.cc:407] Connection header received
0811 09:15:25.049371 (+ 2414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:25.049389 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:25.049795 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:25.050169 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:25.051632 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:25.053400 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:25.055421 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:25.056313 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:25.056473 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.152706 (+1096233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:34793: BlockingRecv error: recv got EOF from 127.17.16.65:34793 (error 108)
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":277,"threads_started":1}
W20260811 09:15:26.155155 403 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:46083 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.16.126:46083: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:15:26.168452 407 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:26.156123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:41393 (local address 127.17.16.126:46083)
0811 09:15:26.156315 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:26.156321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:26.156351 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:15:26.156567 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:26.156573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:26.156663 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:26.156788 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:26.157737 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.158970 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.161167 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.161960 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.162101 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.168160 (+ 6059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:41393: BlockingRecv error: recv got EOF from 127.17.16.65:41393 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:15:26.169317 409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:26.155973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:41393)
0811 09:15:26.156280 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:26.156301 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:26.156419 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:26.156842 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:26.156847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:26.157343 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:26.157575 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:26.157589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.159194 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.159201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:26.160947 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:26.160966 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.162431 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.162439 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:26.162646 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.163333 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:26.163361 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:26.168980 (+ 5619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181}
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:26.182885 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:26.170286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:59877)
0811 09:15:26.171004 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:26.171026 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:26.171247 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:26.171758 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:26.171764 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:26.172383 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:26.172685 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:26.172697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.174313 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.174319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:26.175888 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:26.175905 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.176935 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.176941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:26.177153 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.177839 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:26.177864 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:26.182461 (+ 4597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":411,"threads_started":1}
W20260811 09:15:26.183151 407 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:26.170680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:59877 (local address 127.17.16.126:46083)
0811 09:15:26.171046 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:26.171053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:26.171133 (+ 80us) server_negotiation.cc:407] Connection header received
0811 09:15:26.171395 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:26.171401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:26.171484 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:26.171556 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:26.172865 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.174132 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.176039 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.176723 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.176849 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.182871 (+ 6022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:59877: BlockingRecv error: recv got EOF from 127.17.16.65:59877 (error 108)
Metrics: {"server-negotiator.queue_time_us":232}
W20260811 09:15:26.183561 403 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:15:26.203542 617 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:44385
I20260811 09:15:26.203588 837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:44385 every 8 connection(s)
I20260811 09:15:26.205092 617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:15:26.208974 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 617
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:26.234207 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:26.210640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50014 (local address 127.17.16.126:46083)
0811 09:15:26.216495 (+ 5855us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:26.216500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:26.216515 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:15:26.216565 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:26.216569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:26.216630 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:26.216726 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:26.218342 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.219505 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.221801 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.222626 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.222766 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.226554 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:26.226565 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:26.226584 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:26.226643 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:26.230298 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:26.231263 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:26.231273 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:26.231282 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:26.231490 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:26.231897 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:26.231903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:26.231906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:26.232568 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:26.232919 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:26.233406 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 09:15:26.233777 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":5754,"thread_start_us":5610,"threads_started":1}
W20260811 09:15:26.632370 615 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.087s user 0.001s sys 0.017s
W20260811 09:15:26.633916 616 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:25.527996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:57389 (local address 127.17.16.126:46083)
0811 09:15:25.537618 (+ 9622us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:25.537625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:25.537649 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:15:25.537719 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:25.537725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:25.537805 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:25.537922 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:25.540077 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:25.540859 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:25.542549 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:25.543182 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:25.543332 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.633663 (+1090331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:57389: BlockingRecv error: recv got EOF from 127.17.16.66:57389 (error 108)
Metrics: {"server-negotiator.queue_time_us":9496,"thread_start_us":9370,"threads_started":1}
I20260811 09:15:26.634886 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:25.527380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:57389)
0811 09:15:25.528357 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:25.528435 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:25.530927 (+ 2492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:25.538383 (+ 7456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:25.538396 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:25.539113 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:25.539878 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:25.539900 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:25.541062 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:25.541066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:25.542382 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:25.542397 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:25.543470 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:25.543479 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:25.543823 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:25.544824 (+ 1001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:25.544859 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:26.633490 (+1088631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":634,"threads_started":1}
W20260811 09:15:26.635896 613 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:46083 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.16.126:46083: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:26.646821 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:26.636512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:60841)
0811 09:15:26.636755 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:26.636779 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:26.636961 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:26.637274 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:26.637280 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:26.637836 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:26.638083 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:26.638097 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.639100 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.639105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:26.640876 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:26.640919 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.642095 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.642100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:26.642257 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.642920 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:26.642937 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:26.646611 (+ 3674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 09:15:26.647177 616 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:26.636611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:60841 (local address 127.17.16.126:46083)
0811 09:15:26.636765 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:26.636770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:26.636847 (+ 77us) server_negotiation.cc:407] Connection header received
0811 09:15:26.637075 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:26.637079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:26.637143 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:26.637221 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:26.638220 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.638972 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.641102 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.641844 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.641973 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.646890 (+ 4917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:60841: BlockingRecv error: recv got EOF from 127.17.16.66:60841 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:26 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:26.656823 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:26.647559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:57119)
0811 09:15:26.647714 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:26.647732 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:26.647830 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:26.648323 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:26.648326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:26.648591 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:26.648738 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:26.648745 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.650291 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.650295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:26.651244 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:26.651255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.652209 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.652215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:26.652410 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.653130 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:26.653152 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:26.656618 (+ 3466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 09:15:26.657117 613 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:15:26.657241 616 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:26.647831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:57119 (local address 127.17.16.126:46083)
0811 09:15:26.648007 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:26.648014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:26.648032 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:15:26.648090 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:26.648095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:26.648189 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:26.648300 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:26.648853 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.650149 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.651362 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.652060 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.652185 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.656942 (+ 4757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:57119: BlockingRecv error: recv got EOF from 127.17.16.66:57119 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 09:15:27 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:27 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:27.199049 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:27.184088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:51253)
0811 09:15:27.185128 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:27.185149 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:27.185277 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:27.185658 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:27.185663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:27.186158 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:27.186359 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:27.186372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.187484 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.187490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:27.189026 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:27.189043 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.189747 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.189753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:27.189928 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:27.194914 (+ 4986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:27.194939 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:27.198723 (+ 3784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":907,"thread_start_us":757,"threads_started":1}
W20260811 09:15:27.199712 867 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:27.184195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:51253 (local address 127.17.16.126:46083)
0811 09:15:27.185370 (+ 1175us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:27.185374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:27.185389 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:15:27.185436 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:27.185439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:27.185511 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:27.185598 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:27.186483 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.187315 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.189172 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.189599 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.189701 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:27.198952 (+ 9251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:51253: BlockingRecv error: recv got EOF from 127.17.16.65:51253 (error 108)
Metrics: {"server-negotiator.queue_time_us":1086,"thread_start_us":989,"threads_started":1}
W20260811 09:15:27.235289 856 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.012s user 0.003s sys 0.013s
W20260811 09:15:27.236403 407 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:26.208009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:41969 (local address 127.17.16.126:46083)
0811 09:15:26.208255 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:26.208263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:26.213500 (+ 5237us) server_negotiation.cc:407] Connection header received
0811 09:15:26.216111 (+ 2611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:26.216117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:26.216211 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:26.216322 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:26.218710 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.219709 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.221693 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.222446 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.222551 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:27.236197 (+1013646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:41969: BlockingRecv error: recv got EOF from 127.17.16.67:41969 (error 108)
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":86}
I20260811 09:15:27.236996 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:26.208654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:41969)
0811 09:15:26.213341 (+ 4687us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:26.213396 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:26.215850 (+ 2454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:26.216740 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:26.216753 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:26.217608 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:26.218500 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:26.218525 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.219866 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.219872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:26.221543 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:26.221559 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:26.222679 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:26.222688 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:26.222907 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:26.223524 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:26.223557 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:27.236052 (+1012495us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4400,"thread_start_us":4177,"threads_started":1}
W20260811 09:15:27.237937 841 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:46083 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.16.126:46083: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Aug 11 09:15:27 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:27 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:27.246582 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:27.238539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:43545)
0811 09:15:27.238728 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:27.238744 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:27.238836 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:27.239085 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:27.239088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:27.239415 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:27.239577 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:27.239586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.240560 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.240563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:27.241552 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:27.241567 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.242237 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.242242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:27.242358 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:27.242903 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:27.242920 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:27.246315 (+ 3395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 09:15:27.246770 407 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:27.238653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:43545 (local address 127.17.16.126:46083)
0811 09:15:27.238857 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:27.238862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:27.238877 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:15:27.238920 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:27.238923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:27.238980 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:27.239066 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:27.239703 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.240455 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.241681 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.242093 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.242183 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:27.246510 (+ 4327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:43545: BlockingRecv error: recv got EOF from 127.17.16.67:43545 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 11 09:15:27 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:27 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:27.254732 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:27.247235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:53369)
0811 09:15:27.247400 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:27.247417 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:27.247501 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:27.247827 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:27.247831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:27.248130 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:27.248277 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:27.248284 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.249283 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.249288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:27.250214 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:27.250226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.250895 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.250900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:27.251077 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:27.251557 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:27.251573 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:27.254452 (+ 2879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 09:15:27.254866 407 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:27.247319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:53369 (local address 127.17.16.126:46083)
0811 09:15:27.247452 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:27.247456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:27.247469 (+ 13us) server_negotiation.cc:407] Connection header received
0811 09:15:27.247618 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:27.247624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:27.247701 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:27.247777 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:27.248406 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.249152 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.250344 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.250756 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.250860 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:27.254659 (+ 3799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:53369: BlockingRecv error: recv got EOF from 127.17.16.67:53369 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 09:15:27.255061 841 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 09:15:27 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:27 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:27.666275 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:27.657659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:44691)
0811 09:15:27.658146 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:27.658166 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:27.658278 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:27.658552 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:27.658555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:27.658976 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:27.659125 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:27.659133 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.660072 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.660075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:27.661080 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:27.661096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.662015 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.662019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:27.662144 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:27.662720 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:27.662738 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:27.666038 (+ 3300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":276,"threads_started":1}
W20260811 09:15:27.666446 407 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:27.657751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:44691 (local address 127.17.16.126:46083)
0811 09:15:27.657966 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:27.657972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:27.658237 (+ 265us) server_negotiation.cc:407] Connection header received
0811 09:15:27.658365 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:27.658368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:27.658445 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:27.658531 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:27.659228 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.659966 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.661254 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:27.661716 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:27.661789 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:27.666230 (+ 4441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:44691: BlockingRecv error: recv got EOF from 127.17.16.66:44691 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 09:15:28 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:28 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:28.209148 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:28.200058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:49865)
0811 09:15:28.200647 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:28.200668 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:28.200805 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:28.201192 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:28.201197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:28.201653 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:28.201863 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:28.201874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.202907 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.202912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:28.203925 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:28.203942 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.204924 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.204932 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:28.205081 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:28.205685 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:28.205704 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:28.208835 (+ 3131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":279,"threads_started":1}
W20260811 09:15:28.209292 873 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:28.200159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:49865 (local address 127.17.16.126:46083)
0811 09:15:28.200697 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:28.200703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:28.200724 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:15:28.200940 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:28.200946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:28.201046 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:28.201150 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:28.202006 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.202757 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.204084 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.204749 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.204883 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:28.209072 (+ 4189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:49865: BlockingRecv error: recv got EOF from 127.17.16.65:49865 (error 108)
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":323,"threads_started":1}
Aug 11 09:15:28 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:28 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:28.264401 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:28.255594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:43169)
0811 09:15:28.256077 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:28.256094 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:28.256223 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:28.256488 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:28.256491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:28.256955 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:28.257118 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:28.257129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.258103 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.258107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:28.259037 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:28.259048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.259933 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.259938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:28.260062 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:28.260759 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:28.260782 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:28.264083 (+ 3301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":265,"threads_started":1}
W20260811 09:15:28.264488 873 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:28.255784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:43169 (local address 127.17.16.126:46083)
0811 09:15:28.255999 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:28.256006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:28.256162 (+ 156us) server_negotiation.cc:407] Connection header received
0811 09:15:28.256312 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:28.256316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:28.256379 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:28.256455 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:28.257256 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.257991 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.259161 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.259755 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.259865 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:28.264286 (+ 4421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:43169: BlockingRecv error: recv got EOF from 127.17.16.67:43169 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 09:15:28 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:28 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:28.675726 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:28.667161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:56841)
0811 09:15:28.667625 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:28.667653 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:28.667766 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:28.668035 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:28.668038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:28.668475 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:28.668697 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:28.668712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.670029 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.670033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:28.670945 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:28.670957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.671642 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.671646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:28.671763 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:28.672294 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:28.672309 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:28.675483 (+ 3174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":261,"threads_started":1}
W20260811 09:15:28.675978 873 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:28.667216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:56841 (local address 127.17.16.126:46083)
0811 09:15:28.667427 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:28.667435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:28.667722 (+ 287us) server_negotiation.cc:407] Connection header received
0811 09:15:28.667852 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:28.667856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:28.667918 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:28.667995 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:28.668868 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.669874 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.671068 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:28.671514 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:28.671617 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:28.675695 (+ 4078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:56841: BlockingRecv error: recv got EOF from 127.17.16.66:56841 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 09:15:29 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:29 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:29.218371 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:29.210177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:41797)
0811 09:15:29.210706 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:29.210722 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:29.210827 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:29.211119 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:29.211122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:29.211499 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:29.211660 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:29.211670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.212648 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.212651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:29.213586 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:29.213599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.214281 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.214285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:29.214411 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:29.214938 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:29.214955 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:29.218151 (+ 3196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":268,"threads_started":1}
W20260811 09:15:29.218641 878 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:29.210289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:41797 (local address 127.17.16.126:46083)
0811 09:15:29.210799 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:29.210804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:29.210823 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:15:29.210923 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:29.210927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:29.211002 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:29.211099 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:29.211776 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.212538 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.213735 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.214167 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.214271 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:29.218361 (+ 4090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:41797: BlockingRecv error: recv got EOF from 127.17.16.65:41797 (error 108)
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":279,"threads_started":1}
Aug 11 09:15:29 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:29 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:29.274293 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:29.265339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:47937)
0811 09:15:29.265873 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:29.265912 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:29.266050 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:29.266437 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:29.266442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:29.266957 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:29.267198 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:29.267212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.268298 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.268303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:29.269289 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:29.269303 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.269954 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.269959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:29.270100 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:29.270643 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:29.270661 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:29.274049 (+ 3388us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":315,"threads_started":1}
W20260811 09:15:29.274413 878 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:29.265417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:47937 (local address 127.17.16.126:46083)
0811 09:15:29.265573 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:29.265578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:29.266008 (+ 430us) server_negotiation.cc:407] Connection header received
0811 09:15:29.266192 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:29.266196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:29.266274 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:29.266385 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:29.267343 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.268161 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.269400 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.269796 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.269885 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:29.274228 (+ 4343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:47937: BlockingRecv error: recv got EOF from 127.17.16.67:47937 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 09:15:29 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:29 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:29.685418 885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:29.676761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:56375)
0811 09:15:29.677318 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:29.677340 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:29.677477 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:29.677847 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:29.677851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:29.678255 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:29.678420 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:29.678429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.679452 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.679457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:29.680368 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:29.680380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.681147 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.681153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:29.681343 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:29.681869 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:29.681886 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:29.685080 (+ 3194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":301,"threads_started":1}
W20260811 09:15:29.685566 878 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:29.676791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:56375 (local address 127.17.16.126:46083)
0811 09:15:29.677002 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:29.677008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:29.677426 (+ 418us) server_negotiation.cc:407] Connection header received
0811 09:15:29.677634 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:29.677640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:29.677709 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:29.677790 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:29.678574 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.679305 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.680509 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:29.680987 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:29.681103 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:29.685323 (+ 4220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:56375: BlockingRecv error: recv got EOF from 127.17.16.66:56375 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 09:15:30 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:30 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:30.228474 887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:30.219378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:59209)
0811 09:15:30.219933 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:30.219954 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:30.220085 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:30.220418 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:30.220423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:30.220992 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:30.221188 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:30.221199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.222279 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.222285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:30.223471 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:30.223488 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.224334 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.224338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:30.224510 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:30.225153 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:30.225170 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:30.228231 (+ 3061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":257,"threads_started":1}
W20260811 09:15:30.228725 888 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:30.219471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:59209 (local address 127.17.16.126:46083)
0811 09:15:30.220042 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:30.220047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:30.220065 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:15:30.220202 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:30.220207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:30.220278 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:30.220366 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:30.221328 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.222124 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.223598 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.224181 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.224327 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:30.228449 (+ 4122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:59209: BlockingRecv error: recv got EOF from 127.17.16.65:59209 (error 108)
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":361,"threads_started":1}
Aug 11 09:15:30 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:30 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:30.284037 892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:30.275161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:57415)
0811 09:15:30.275650 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:30.275665 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:30.275793 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:30.276117 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:30.276123 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:30.276507 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:30.276681 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:30.276691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.277759 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.277763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:30.278664 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:30.278676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.279339 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.279346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:30.279506 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:30.280041 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:30.280058 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:30.283710 (+ 3652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":247,"threads_started":1}
W20260811 09:15:30.284159 888 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:30.275263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:57415 (local address 127.17.16.126:46083)
0811 09:15:30.275464 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:30.275470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:30.275746 (+ 276us) server_negotiation.cc:407] Connection header received
0811 09:15:30.275903 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:30.275908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:30.275972 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:30.276052 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:30.276836 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.277615 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.278800 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.279205 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.279310 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:30.283946 (+ 4636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:57415: BlockingRecv error: recv got EOF from 127.17.16.67:57415 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 09:15:30 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:30 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:30.695284 893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:30.686643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:48161)
0811 09:15:30.687171 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:30.687194 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:30.687307 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:30.687623 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:30.687626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:30.687991 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:30.688167 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:30.688177 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.689251 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.689255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:30.690517 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:30.690541 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.691193 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.691197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:30.691328 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:30.691876 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:30.691893 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:30.695040 (+ 3147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":276,"threads_started":1}
W20260811 09:15:30.695446 888 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:30.686753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:48161 (local address 127.17.16.126:46083)
0811 09:15:30.686928 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:30.686932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:30.687285 (+ 353us) server_negotiation.cc:407] Connection header received
0811 09:15:30.687412 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:30.687415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:30.687478 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:30.687557 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:30.688278 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.689106 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.690652 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:30.691065 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:30.691168 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:30.695221 (+ 4053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:48161: BlockingRecv error: recv got EOF from 127.17.16.66:48161 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 09:15:31 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:31 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:31.237866 894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:31.229383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:44705)
0811 09:15:31.229911 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:31.229928 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:31.230062 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:31.230364 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:31.230368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:31.230812 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:31.230974 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:31.230985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.232007 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.232011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:31.232976 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:31.232989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.233601 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.233606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:31.233728 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:31.234223 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:31.234240 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:31.237539 (+ 3299us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":276,"threads_started":1}
W20260811 09:15:31.238020 895 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:31.229439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:44705 (local address 127.17.16.126:46083)
0811 09:15:31.229878 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:31.229882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:31.229979 (+ 97us) server_negotiation.cc:407] Connection header received
0811 09:15:31.230176 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:31.230179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:31.230254 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:31.230335 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:31.231092 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.231886 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.233083 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.233470 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.233557 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:31.237772 (+ 4215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:44705: BlockingRecv error: recv got EOF from 127.17.16.65:44705 (error 108)
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":208,"threads_started":1}
Aug 11 09:15:31 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:31 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:31.293691 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:31.285055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:36507)
0811 09:15:31.285576 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:31.285592 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:31.285712 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:31.286074 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:31.286078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:31.286555 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:31.286748 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:31.286758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.287845 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.287850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:31.288839 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:31.288852 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.289518 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.289522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:31.289653 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:31.290199 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:31.290217 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:31.293438 (+ 3221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":268,"threads_started":1}
W20260811 09:15:31.293850 895 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:31.285080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:36507 (local address 127.17.16.126:46083)
0811 09:15:31.285271 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:31.285276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:31.285703 (+ 427us) server_negotiation.cc:407] Connection header received
0811 09:15:31.285822 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:31.285826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:31.285896 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:31.285993 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:31.286883 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.287705 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.289015 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.289409 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.289504 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:31.293640 (+ 4136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:36507: BlockingRecv error: recv got EOF from 127.17.16.67:36507 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 09:15:31 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:31 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:31.705404 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:31.696136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:36225)
0811 09:15:31.696727 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:31.696755 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:31.696928 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:31.697306 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:31.697312 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:31.697795 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:31.697970 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:31.697982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.699027 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.699034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:31.700032 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:31.700046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.700826 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.700832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:31.701062 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:31.701653 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:31.701673 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:31.705105 (+ 3432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":318,"threads_started":1}
W20260811 09:15:31.705571 895 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:31.696342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:36225 (local address 127.17.16.126:46083)
0811 09:15:31.696584 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:31.696591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:31.696845 (+ 254us) server_negotiation.cc:407] Connection header received
0811 09:15:31.697067 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:31.697072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:31.697157 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:31.697254 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:31.698134 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.698880 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.700187 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:31.700661 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:31.700780 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:31.705334 (+ 4554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:36225: BlockingRecv error: recv got EOF from 127.17.16.66:36225 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 09:15:32 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:32 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:32.257215 898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:32.238916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:53799)
0811 09:15:32.239398 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:32.239415 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:32.239519 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:32.239823 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:32.239826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:32.240228 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:32.240392 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:32.240401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.241399 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.241404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:32.242339 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:32.242352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.243178 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.243182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:32.243324 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:32.253162 (+ 9838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:32.253189 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:32.256862 (+ 3673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":235,"threads_started":1}
W20260811 09:15:32.257359 899 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:32.238985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:53799 (local address 127.17.16.126:46083)
0811 09:15:32.239473 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:32.239477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:32.239494 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:15:32.239631 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:32.239634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:32.239711 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:32.239797 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:32.240498 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.241273 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.242476 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.243033 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.243155 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:32.257113 (+ 13958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:53799: BlockingRecv error: recv got EOF from 127.17.16.65:53799 (error 108)
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":265,"threads_started":1}
Aug 11 09:15:32 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:32 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:32.303638 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:32.294539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:55049)
0811 09:15:32.295086 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:32.295105 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:32.295247 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:32.295626 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:32.295632 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:32.296114 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:32.296282 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:32.296293 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.297438 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.297446 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:32.298436 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:32.298451 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.299192 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.299199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:32.299376 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:32.299928 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:32.299961 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:32.303421 (+ 3460us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":268,"threads_started":1}
W20260811 09:15:32.303880 899 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:32.294679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:55049 (local address 127.17.16.126:46083)
0811 09:15:32.294892 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:32.294899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:32.295194 (+ 295us) server_negotiation.cc:407] Connection header received
0811 09:15:32.295371 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:32.295377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:32.295473 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:32.295570 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:32.296450 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.297292 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.298588 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.299032 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.299152 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:32.303631 (+ 4479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:55049: BlockingRecv error: recv got EOF from 127.17.16.67:55049 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 09:15:32 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:32 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:32.715049 904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:32.706360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:49457)
0811 09:15:32.706873 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:32.706895 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:32.707012 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:32.707314 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:32.707317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:32.707735 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:32.707913 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:32.707922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.709096 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.709102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:32.710201 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:32.710216 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.710859 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.710863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:32.710987 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:32.711531 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:32.711548 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:32.714821 (+ 3273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":277,"threads_started":1}
W20260811 09:15:32.715380 899 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:32.706442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:49457 (local address 127.17.16.126:46083)
0811 09:15:32.706595 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:32.706600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:32.706959 (+ 359us) server_negotiation.cc:407] Connection header received
0811 09:15:32.707114 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:32.707118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:32.707199 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:32.707283 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:32.708049 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.708961 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.710325 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:32.710734 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:32.710830 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:32.715078 (+ 4248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:49457: BlockingRecv error: recv got EOF from 127.17.16.66:49457 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 09:15:33 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:33 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:33.266505 906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:33.258088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:33191)
0811 09:15:33.258662 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:33.258678 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:33.258788 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:33.259054 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:33.259057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:33.259447 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:33.259639 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:33.259649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.260569 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.260572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:33.261546 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:33.261558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.262230 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.262235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:33.262350 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:33.262898 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:33.262914 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:33.266200 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":265,"threads_started":1}
W20260811 09:15:33.266672 905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:33.258133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:33191 (local address 127.17.16.126:46083)
0811 09:15:33.258610 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:33.258616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:33.258733 (+ 117us) server_negotiation.cc:407] Connection header received
0811 09:15:33.258864 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:33.258868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:33.258940 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:33.259025 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:33.259747 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.260464 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.261654 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.262091 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.262187 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:33.266434 (+ 4247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:33191: BlockingRecv error: recv got EOF from 127.17.16.65:33191 (error 108)
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":249,"threads_started":1}
Aug 11 09:15:33 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:33 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:33.312786 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:33.304432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:36729)
0811 09:15:33.304949 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:33.304964 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:33.305091 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:33.305412 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:33.305416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:33.305848 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:33.306014 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:33.306023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.307047 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.307051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:33.307967 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:33.307979 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.308722 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.308728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:33.308868 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:33.309414 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:33.309433 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:33.312546 (+ 3113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":269,"threads_started":1}
W20260811 09:15:33.312995 905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:33.304571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:36729 (local address 127.17.16.126:46083)
0811 09:15:33.304775 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:33.304780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:33.305038 (+ 258us) server_negotiation.cc:407] Connection header received
0811 09:15:33.305213 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:33.305218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:33.305284 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:33.305362 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:33.306151 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.306900 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.308100 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.308559 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.308702 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:33.312776 (+ 4074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:36729: BlockingRecv error: recv got EOF from 127.17.16.67:36729 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 09:15:33 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:33 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:33.725109 909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:33.715963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:34685)
0811 09:15:33.716613 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:33.716633 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:33.716780 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:33.717126 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:33.717130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:33.717572 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:33.717745 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:33.717754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.718784 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.718790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:33.719769 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:33.719783 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.720498 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.720504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:33.720670 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:33.721342 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:33.721360 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:33.724813 (+ 3453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":316,"threads_started":1}
W20260811 09:15:33.725263 905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:33.715979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:34685 (local address 127.17.16.126:46083)
0811 09:15:33.716233 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:33.716239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:33.716721 (+ 482us) server_negotiation.cc:407] Connection header received
0811 09:15:33.716919 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:33.716925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:33.716994 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:33.717075 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:33.717885 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.718645 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.719906 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:33.720349 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:33.720454 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:33.725042 (+ 4588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:34685: BlockingRecv error: recv got EOF from 127.17.16.66:34685 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 11 09:15:34 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:34 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:34.276376 912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:34.267557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:57961)
0811 09:15:34.268087 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:34.268104 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:34.268212 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:34.268614 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:34.268617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:34.269068 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:34.269250 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:34.269261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.270290 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.270293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:34.271214 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:34.271226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.271891 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.271895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:34.272011 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:34.272593 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:34.272610 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:34.276127 (+ 3517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":258,"threads_started":1}
W20260811 09:15:34.276669 913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:34.267671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:57961 (local address 127.17.16.126:46083)
0811 09:15:34.268375 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:34.268380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:34.268394 (+ 14us) server_negotiation.cc:407] Connection header received
0811 09:15:34.268438 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:34.268442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:34.268499 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:34.268586 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:34.269383 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.270184 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.271329 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.271757 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.271846 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:34.276385 (+ 4539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:57961: BlockingRecv error: recv got EOF from 127.17.16.65:57961 (error 108)
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":523,"threads_started":1}
Aug 11 09:15:34 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:34 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:34.321920 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:34.313570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:51523)
0811 09:15:34.314160 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:34.314175 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:34.314300 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:34.314576 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:34.314580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:34.314967 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:34.315154 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:34.315163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.316083 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.316086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:34.317065 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:34.317077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.317741 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.317746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:34.317862 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:34.318418 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:34.318436 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:34.321676 (+ 3240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":257,"threads_started":1}
W20260811 09:15:34.322170 913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:34.313731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:51523 (local address 127.17.16.126:46083)
0811 09:15:34.313899 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:34.313903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:34.314244 (+ 341us) server_negotiation.cc:407] Connection header received
0811 09:15:34.314391 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:34.314395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:34.314458 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:34.314533 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:34.315261 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.315974 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.317191 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.317599 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.317686 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:34.321838 (+ 4152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:51523: BlockingRecv error: recv got EOF from 127.17.16.67:51523 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 09:15:34 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:34 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:34.735008 918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:34.726061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:58477)
0811 09:15:34.726570 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:34.726590 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:34.726716 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:34.726991 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:34.726994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:34.727399 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:34.727568 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:34.727577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.728808 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.728813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:34.730131 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:34.730148 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.730867 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.730872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:34.730987 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:34.731522 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:34.731538 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:34.734751 (+ 3213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":252,"threads_started":1}
W20260811 09:15:34.735145 913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:34.726110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:58477 (local address 127.17.16.126:46083)
0811 09:15:34.726269 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:34.726273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:34.726655 (+ 382us) server_negotiation.cc:407] Connection header received
0811 09:15:34.726804 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:34.726808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:34.726883 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:34.726962 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:34.727674 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.728661 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.730256 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:34.730710 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:34.730797 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:34.734938 (+ 4141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:58477: BlockingRecv error: recv got EOF from 127.17.16.66:58477 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 09:15:35 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:35 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:35.285701 920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:35.277359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:38813)
0811 09:15:35.277897 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:35.277913 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:35.278033 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:35.278318 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:35.278321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:35.278730 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:35.278888 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:35.278898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.279825 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.279829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:35.280767 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:35.280779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.281423 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.281429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:35.281580 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:35.282164 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:35.282181 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:35.285471 (+ 3290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":315,"threads_started":1}
W20260811 09:15:35.285899 921 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:35.277431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:38813 (local address 127.17.16.126:46083)
0811 09:15:35.277867 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:35.277871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:35.277966 (+ 95us) server_negotiation.cc:407] Connection header received
0811 09:15:35.278126 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:35.278129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:35.278203 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:35.278282 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:35.278996 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.279720 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.280871 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.281292 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.281380 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:35.285677 (+ 4297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:38813: BlockingRecv error: recv got EOF from 127.17.16.65:38813 (error 108)
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":238,"threads_started":1}
Aug 11 09:15:35 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:35 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:35.331220 922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:35.322748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:46381)
0811 09:15:35.323293 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:35.323309 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:35.323420 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:35.323665 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:35.323670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:35.324120 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:35.324292 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:35.324301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.325458 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.325462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:35.326387 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:35.326399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.327062 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.327066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:35.327180 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:35.327686 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:35.327702 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:35.330970 (+ 3268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":292,"threads_started":1}
W20260811 09:15:35.331341 921 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:35.322798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:46381 (local address 127.17.16.126:46083)
0811 09:15:35.322949 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:35.322953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:35.323369 (+ 416us) server_negotiation.cc:407] Connection header received
0811 09:15:35.323500 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:35.323503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:35.323563 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:35.323643 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:35.324410 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.325342 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.326493 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.326912 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.326999 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:35.331148 (+ 4149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:46381: BlockingRecv error: recv got EOF from 127.17.16.67:46381 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 09:15:35 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:35 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:35.744585 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:35.735926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:45893)
0811 09:15:35.736460 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:35.736481 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:35.736599 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:35.736881 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:35.736884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:35.737318 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:35.737473 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:35.737483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.738454 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.738458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:35.739419 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:35.739432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.740165 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.740171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:35.740305 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:35.741015 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:35.741033 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:35.744355 (+ 3322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":283,"threads_started":1}
W20260811 09:15:35.744764 921 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:35.735984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:45893 (local address 127.17.16.126:46083)
0811 09:15:35.736137 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:35.736142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:35.736557 (+ 415us) server_negotiation.cc:407] Connection header received
0811 09:15:35.736689 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:35.736693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:35.736767 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:35.736849 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:35.737612 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.738326 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.739562 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:35.740006 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:35.740094 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:35.744547 (+ 4453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:45893: BlockingRecv error: recv got EOF from 127.17.16.66:45893 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 09:15:36 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:36 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:36.295423 926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:36.286580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:36327)
0811 09:15:36.287305 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:36.287321 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:36.287432 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:36.287793 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:36.287799 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:36.288279 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:36.288447 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:36.288458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.289520 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.289526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:36.290491 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:36.290504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.291195 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.291202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:36.291365 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:36.291915 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:36.291932 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:36.295103 (+ 3171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":485,"threads_started":1}
W20260811 09:15:36.295595 927 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:36.286653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:36327 (local address 127.17.16.126:46083)
0811 09:15:36.287247 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:36.287253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:36.287389 (+ 136us) server_negotiation.cc:407] Connection header received
0811 09:15:36.287561 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:36.287565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:36.287628 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:36.287723 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:36.288603 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.289372 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.290631 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.291049 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.291155 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:36.295325 (+ 4170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:36327: BlockingRecv error: recv got EOF from 127.17.16.65:36327 (error 108)
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":239,"threads_started":1}
Aug 11 09:15:36 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:36 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:36.340591 929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:36.332101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:55915)
0811 09:15:36.332535 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:36.332551 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:36.332661 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:36.332969 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:36.332972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:36.333488 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:36.333677 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:36.333691 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.334909 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.334913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:36.335854 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:36.335866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.336594 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.336599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:36.336728 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:36.337291 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:36.337308 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:36.340364 (+ 3056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":229,"threads_started":1}
W20260811 09:15:36.340736 927 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:36.332151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:55915 (local address 127.17.16.126:46083)
0811 09:15:36.332337 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:36.332344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:36.332626 (+ 282us) server_negotiation.cc:407] Connection header received
0811 09:15:36.332748 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:36.332751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:36.332815 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:36.332940 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:36.333823 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.334793 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.336026 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.336449 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.336562 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:36.340537 (+ 3975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:55915: BlockingRecv error: recv got EOF from 127.17.16.67:55915 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 09:15:36 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:36 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:36.754209 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:36.745439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:36847)
0811 09:15:36.745970 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:36.745991 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:36.746109 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:36.746400 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:36.746403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:36.746839 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:36.747029 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:36.747040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.748048 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.748052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:36.749064 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:36.749076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.749764 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.749768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:36.749871 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:36.750459 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:36.750480 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:36.753924 (+ 3444us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":297,"threads_started":1}
W20260811 09:15:36.754304 927 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:36.745490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:36847 (local address 127.17.16.126:46083)
0811 09:15:36.745672 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:36.745679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:36.746066 (+ 387us) server_negotiation.cc:407] Connection header received
0811 09:15:36.746219 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:36.746223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:36.746298 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:36.746372 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:36.747165 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.747931 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.749209 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:36.749647 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:36.749743 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:36.754099 (+ 4356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:36847: BlockingRecv error: recv got EOF from 127.17.16.66:36847 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 09:15:37 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:37 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:37.308971 932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:37.296372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:57417)
0811 09:15:37.296876 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:37.296918 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:37.297049 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:37.297308 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:37.297310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:37.297720 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:37.297892 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:37.297900 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.298865 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.298869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:37.299898 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:37.299912 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.300848 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.300853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:37.301002 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:37.305482 (+ 4480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:37.305500 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:37.308720 (+ 3220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":230,"threads_started":1}
W20260811 09:15:37.309249 934 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:37.296603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:57417 (local address 127.17.16.126:46083)
0811 09:15:37.297060 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:37.297065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:37.297081 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:15:37.297128 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:37.297132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:37.297204 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:37.297288 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:37.298010 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.298744 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.300030 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.300695 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.300814 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:37.308954 (+ 8140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:57417: BlockingRecv error: recv got EOF from 127.17.16.65:57417 (error 108)
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":232,"threads_started":1}
Aug 11 09:15:37 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:37 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:37.349768 935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:37.341450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:53057)
0811 09:15:37.341962 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:37.341979 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:37.342092 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:37.342359 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:37.342362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:37.342738 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:37.342911 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:37.342920 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.343840 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.343843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:37.344812 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:37.344825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.345591 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.345599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:37.345755 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:37.346353 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:37.346371 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:37.349534 (+ 3163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":280,"threads_started":1}
W20260811 09:15:37.349887 934 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:37.341523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:53057 (local address 127.17.16.126:46083)
0811 09:15:37.341690 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:37.341694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:37.342054 (+ 360us) server_negotiation.cc:407] Connection header received
0811 09:15:37.342176 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:37.342180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:37.342251 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:37.342328 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:37.343032 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.343739 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.344939 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.345396 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.345506 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:37.349708 (+ 4202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:53057: BlockingRecv error: recv got EOF from 127.17.16.67:53057 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 09:15:37 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:37 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:37.763620 937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:37.755041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:60625)
0811 09:15:37.755541 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:37.755562 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:37.755690 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:37.756023 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:37.756028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:37.756457 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:37.756652 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:37.756662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.757779 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.757785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:37.758701 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:37.758713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.759421 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.759428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:37.759599 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:37.760183 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:37.760200 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:37.763287 (+ 3087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":267,"threads_started":1}
W20260811 09:15:37.763754 934 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:37.755131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:60625 (local address 127.17.16.126:46083)
0811 09:15:37.755307 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:37.755313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:37.755637 (+ 324us) server_negotiation.cc:407] Connection header received
0811 09:15:37.755806 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:37.755811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:37.755891 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:37.755971 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:37.756786 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.757652 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.758836 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:37.759277 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:37.759383 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:37.763488 (+ 4105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:60625: BlockingRecv error: recv got EOF from 127.17.16.66:60625 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 09:15:38 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:38 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:38.318842 941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:38.309908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:47137)
0811 09:15:38.310468 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:38.310485 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:38.310601 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:38.310881 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:38.310884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:38.311293 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:38.311466 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:38.311481 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.312619 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.312625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:38.313914 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:38.313933 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.314651 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.314655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:38.314778 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:38.315328 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:38.315344 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:38.318603 (+ 3259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":300,"threads_started":1}
W20260811 09:15:38.318985 944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:38.309984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:47137 (local address 127.17.16.126:46083)
0811 09:15:38.310470 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:38.310474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:38.310542 (+ 68us) server_negotiation.cc:407] Connection header received
0811 09:15:38.310691 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:38.310695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:38.310770 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:38.310855 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:38.311598 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.312465 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.314065 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.314487 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.314603 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:38.318791 (+ 4188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:47137: BlockingRecv error: recv got EOF from 127.17.16.65:47137 (error 108)
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":264,"threads_started":1}
Aug 11 09:15:38 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:38 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:38.359162 946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:38.350588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:51259)
0811 09:15:38.351054 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:38.351070 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:38.351198 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:38.351506 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:38.351511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:38.351939 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:38.352127 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:38.352137 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.353154 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.353158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:38.354066 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:38.354079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.354762 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.354768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:38.354930 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:38.355493 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:38.355515 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:38.358859 (+ 3344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":247,"threads_started":1}
W20260811 09:15:38.359302 944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:38.350679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:51259 (local address 127.17.16.126:46083)
0811 09:15:38.350845 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:38.350850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:38.351150 (+ 300us) server_negotiation.cc:407] Connection header received
0811 09:15:38.351320 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:38.351323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:38.351382 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:38.351461 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:38.352265 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.353022 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.354202 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.354616 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.354720 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:38.359066 (+ 4346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:51259: BlockingRecv error: recv got EOF from 127.17.16.67:51259 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 09:15:38 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:38 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:38.773501 947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:38.764516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:41333)
0811 09:15:38.765163 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:38.765184 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:38.765302 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:38.765610 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:38.765613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:38.766008 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:38.766219 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:38.766229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.767572 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.767579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:38.768626 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:38.768640 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.769433 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.769438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:38.769573 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:38.770150 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:38.770166 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:38.773263 (+ 3097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":354,"threads_started":1}
W20260811 09:15:38.773674 944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:38.764652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:41333 (local address 127.17.16.126:46083)
0811 09:15:38.764822 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:38.764827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:38.765252 (+ 425us) server_negotiation.cc:407] Connection header received
0811 09:15:38.765411 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:38.765415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:38.765485 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:38.765575 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:38.766364 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.767420 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.768775 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:38.769263 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:38.769361 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:38.773438 (+ 4077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:41333: BlockingRecv error: recv got EOF from 127.17.16.66:41333 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 09:15:39 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:39 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:39.328711 948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:39.319753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:45761)
0811 09:15:39.320239 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:39.320260 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:39.320394 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:39.320752 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:39.320758 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:39.321269 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:39.321447 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:39.321458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.322506 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.322511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:39.323420 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:39.323432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.324222 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.324228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:39.324412 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:39.325029 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:39.325047 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:39.328403 (+ 3356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":222,"threads_started":1}
W20260811 09:15:39.328881 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:39.319802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:45761 (local address 127.17.16.126:46083)
0811 09:15:39.320471 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:39.320476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:39.320492 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:15:39.320547 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:39.320551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:39.320606 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:39.320693 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:39.321604 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.322358 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.323566 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.324085 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.324191 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:39.328625 (+ 4434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:45761: BlockingRecv error: recv got EOF from 127.17.16.65:45761 (error 108)
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":460,"threads_started":1}
Aug 11 09:15:39 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:39 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:39.368362 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:39.359976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:55879)
0811 09:15:39.360531 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:39.360548 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:39.360674 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:39.360979 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:39.360983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:39.361388 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:39.361560 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:39.361569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.362590 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.362594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:39.363514 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:39.363525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.364217 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.364225 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:39.364397 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:39.365014 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:39.365030 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:39.368121 (+ 3091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":302,"threads_started":1}
W20260811 09:15:39.368463 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:39.360083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:55879 (local address 127.17.16.126:46083)
0811 09:15:39.360330 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:39.360335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:39.360619 (+ 284us) server_negotiation.cc:407] Connection header received
0811 09:15:39.360763 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:39.360767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:39.360829 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:39.360933 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:39.361702 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.362452 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.363645 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.364062 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.364152 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:39.368298 (+ 4146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:55879: BlockingRecv error: recv got EOF from 127.17.16.67:55879 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 11 09:15:39 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:39 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:39.782709 952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:39.774343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:34289)
0811 09:15:39.774839 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:39.774864 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:39.774978 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:39.775257 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:39.775260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:39.775690 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:39.775841 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:39.775851 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.776800 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.776803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:39.777724 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:39.777736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.778357 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.778361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:39.778482 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:39.779083 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:39.779102 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:39.782482 (+ 3380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":260,"threads_started":1}
W20260811 09:15:39.782893 950 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:39.774507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:34289 (local address 127.17.16.126:46083)
0811 09:15:39.774688 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:39.774693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:39.774929 (+ 236us) server_negotiation.cc:407] Connection header received
0811 09:15:39.775067 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:39.775070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:39.775147 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:39.775225 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:39.775950 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.776697 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.777839 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:39.778251 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:39.778340 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:39.782676 (+ 4336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:34289: BlockingRecv error: recv got EOF from 127.17.16.66:34289 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 09:15:40 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:40 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.65@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:40.338747 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:40.330078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.65:46337)
0811 09:15:40.330805 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:40.330822 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:40.330932 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:40.331262 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:40.331266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:40.331781 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:40.331976 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:40.331987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.333007 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.333011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:40.333926 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:40.333938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.334576 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.334579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:40.334710 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:40.335279 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:40.335297 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:40.338489 (+ 3192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":234,"threads_started":1}
W20260811 09:15:40.338900 956 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:40.330151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:46337 (local address 127.17.16.126:46083)
0811 09:15:40.330786 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:40.330792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:40.330889 (+ 97us) server_negotiation.cc:407] Connection header received
0811 09:15:40.331035 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:40.331039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:40.331112 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:40.331225 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:40.332101 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.332847 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.334048 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.334473 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.334569 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:40.338694 (+ 4125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.65:46337: BlockingRecv error: recv got EOF from 127.17.16.65:46337 (error 108)
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":339,"threads_started":1}
Aug 11 09:15:40 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:40 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.67@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:40.377524 960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:40.369251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.67:33909)
0811 09:15:40.369741 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:40.369760 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:40.369872 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:40.370116 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:40.370120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:40.370550 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:40.370702 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:40.370712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.371637 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.371641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:40.372551 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:40.372564 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.373218 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.373222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:40.373348 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:40.373882 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:40.373898 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:40.377197 (+ 3299us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":279,"threads_started":1}
W20260811 09:15:40.377615 956 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:40.369305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:33909 (local address 127.17.16.126:46083)
0811 09:15:40.369546 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:40.369550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:40.369825 (+ 275us) server_negotiation.cc:407] Connection header received
0811 09:15:40.369953 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:40.369957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:40.370013 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:40.370088 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:40.370808 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.371502 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.372669 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.373086 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.373181 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:40.377406 (+ 4225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.67:33909: BlockingRecv error: recv got EOF from 127.17.16.67:33909 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 11 09:15:40 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:15:40 dist-test-slave-ncp9 krb5kdc[31697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.16.66@KRBTEST.COM for oryx/127.17.16.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:15:40.793278 961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:40.783858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46083 (local address 127.17.16.66:50469)
0811 09:15:40.784420 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:40.784474 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:40.784604 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:40.784970 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:40.784975 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:40.785392 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:40.785615 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:40.785626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.786666 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.786672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:40.787978 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:40.787994 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.788692 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.788696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:40.788834 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:40.789476 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:40.789497 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:40.792997 (+ 3500us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.16.126:46083: Server oryx/127.17.16.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":286,"threads_started":1}
W20260811 09:15:40.793363 956 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:40.783940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:50469 (local address 127.17.16.126:46083)
0811 09:15:40.784096 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:40.784100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:40.784542 (+ 442us) server_negotiation.cc:407] Connection header received
0811 09:15:40.784697 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:40.784700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:40.784774 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:40.784856 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:40.785753 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.786501 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.788123 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:40.788559 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:40.788653 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:40.793164 (+ 4511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.16.66:50469: BlockingRecv error: recv got EOF from 127.17.16.66:50469 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:15:41.217149 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 32646
I20260811 09:15:41.230048 964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:41.218257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.17.16.65:45279)
0811 09:15:41.218886 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:41.218894 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:41.218924 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:15:41.218998 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:41.219005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:41.219240 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:41.219409 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:41.220978 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:41.222154 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:41.223360 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:41.224025 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:41.224163 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:41.226616 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:41.226626 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:41.226640 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:41.226686 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:41.229475 (+ 2789us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.16.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 09:15:41.229799 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55896: Request ticket server kudu/127.17.16.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":299,"threads_started":1}
W20260811 09:15:41.230129 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 32646: Not authorized: Client connection negotiation failed: client connection to 127.17.16.65:45279: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.16.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 09:15:41.230190 964 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55896: Request ticket server kudu/127.17.16.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 09:15:41.230333 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 32646
I20260811 09:15:41.251678 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 411
I20260811 09:15:41.263973 968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:41.252839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46294 (local address 127.17.16.66:37983)
0811 09:15:41.253565 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:41.253575 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:41.253612 (+ 37us) server_negotiation.cc:407] Connection header received
0811 09:15:41.253717 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:41.253727 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:41.254044 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:41.254297 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:41.255412 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:41.256471 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:41.257770 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:41.258339 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:41.258437 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:41.261015 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:41.261024 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:41.261037 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:41.261077 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:41.263450 (+ 2373us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.16.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 09:15:41.263731 (+ 281us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46294: Request ticket server kudu/127.17.16.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":328,"threads_started":1}
W20260811 09:15:41.264081 968 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46294: Request ticket server kudu/127.17.16.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 09:15:41.264096 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 411: Not authorized: Client connection negotiation failed: client connection to 127.17.16.66:37983: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.16.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 09:15:41.264272 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 411
I20260811 09:15:41.286228 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 617
I20260811 09:15:41.298429 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:41.287298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35014 (local address 127.17.16.67:44385)
0811 09:15:41.287940 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:41.287948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:41.287973 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:15:41.288037 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:41.288043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:41.288282 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:41.288438 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:41.290303 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:41.291300 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:41.292466 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:41.293052 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:41.293150 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:41.295478 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:41.295487 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:41.295500 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:41.295553 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:41.297907 (+ 2354us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.16.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 09:15:41.298174 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35014: Request ticket server kudu/127.17.16.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":320,"threads_started":1}
W20260811 09:15:41.298533 969 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35014: Request ticket server kudu/127.17.16.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 09:15:41.298555 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 617: Not authorized: Client connection negotiation failed: client connection to 127.17.16.67:44385: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.16.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 09:15:41.298697 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 617
I20260811 09:15:41.322587 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 32534
I20260811 09:15:41.323858 32633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:41.440109 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 32534
2026-08-11T09:15:41Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21436 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T09:15:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:15:41Z Disabled control of system clock
I20260811 09:15:41.528481 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41967
--webserver_interface=127.17.16.126
--webserver_port=0
--builtin_ntp_servers=127.17.16.84:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 09:15:41.859011 978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:41.859272 978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:41.859388 978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:41.866654 978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:15:41.866772 978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:41.866844 978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:41.866925 978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:15:41.867007 978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:15:41.876207 978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41967
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:41.877550 978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:41.879472 978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:41.891469 987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:41.891378 992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:41.891505 988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:41.892216 978 server_base.cc:1034] running on GCE node
I20260811 09:15:41.893004 978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:41.894349 978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:41.895545 978 hybrid_clock.cc:648] HybridClock initialized: now 1786439741895504 us; error 35 us; skew 500 ppm
I20260811 09:15:41.895854 978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:15:41.898000 978 webserver.cc:460] Webserver started at http://127.17.16.126:39933/ using document root <none> and password file <none>
I20260811 09:15:41.898633 978 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:41.898720 978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:41.898991 978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:41.901846 978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "858eba09fe31466fbc3d49dc9bec5b59"
format_stamp: "Formatted at 2026-08-11 09:15:41 on dist-test-slave-ncp9"
server_key: "75c862fcd7fbb109188f264058afb1ef"
server_key_iv: "5d04342b5fbb9ef59bca9e729dca246b"
server_key_version: "encryptionkey@0"
I20260811 09:15:41.902601 978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "858eba09fe31466fbc3d49dc9bec5b59"
format_stamp: "Formatted at 2026-08-11 09:15:41 on dist-test-slave-ncp9"
server_key: "75c862fcd7fbb109188f264058afb1ef"
server_key_iv: "5d04342b5fbb9ef59bca9e729dca246b"
server_key_version: "encryptionkey@0"
I20260811 09:15:41.907476 978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:15:41.910804 1001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:41.912024 978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:15:41.912163 978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "858eba09fe31466fbc3d49dc9bec5b59"
format_stamp: "Formatted at 2026-08-11 09:15:41 on dist-test-slave-ncp9"
server_key: "75c862fcd7fbb109188f264058afb1ef"
server_key_iv: "5d04342b5fbb9ef59bca9e729dca246b"
server_key_version: "encryptionkey@0"
I20260811 09:15:41.912331 978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:41.936623 978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:41.937429 978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:41.937700 978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:41.954960 978 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:41967
I20260811 09:15:41.954980 1083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:41967 every 8 connection(s)
I20260811 09:15:41.956256 978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:15:41.959908 1084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:41.960477 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 978
I20260811 09:15:41.960671 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:15:41.961316 17473 external_mini_cluster.cc:1428] Setting key 5fe248d6fdd19b2332a50c6a72859bc5
I20260811 09:15:41.974076 1084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59: Bootstrap starting.
I20260811 09:15:41.975994 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:41.963407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45130 (local address 127.17.16.126:41967)
0811 09:15:41.964577 (+ 1170us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:41.964591 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:41.964649 (+ 58us) server_negotiation.cc:407] Connection header received
0811 09:15:41.966372 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:41.966399 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:41.966961 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:41.967392 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:41.967894 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:41.969726 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:41.971030 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:41.972057 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:41.972217 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:41.973322 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:41.973333 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:41.973349 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:41.974014 (+ 665us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:41.974134 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:41.974163 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:41.974276 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:41.974468 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 09:15:41.974766 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":328,"threads_started":1}
I20260811 09:15:41.977737 1084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:41.978927 1084 log.cc:824] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:41.981400 1084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59: No bootstrap required, opened a new log
I20260811 09:15:41.988723 1084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858eba09fe31466fbc3d49dc9bec5b59" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41967 } }
I20260811 09:15:41.989107 1084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:41.989183 1084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858eba09fe31466fbc3d49dc9bec5b59, State: Initialized, Role: FOLLOWER
I20260811 09:15:41.989670 1084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858eba09fe31466fbc3d49dc9bec5b59" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41967 } }
I20260811 09:15:41.989903 1084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:15:41.990058 1084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:15:41.990232 1084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:41.992525 1084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858eba09fe31466fbc3d49dc9bec5b59" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41967 } }
I20260811 09:15:41.993057 1084 leader_election.cc:302] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 858eba09fe31466fbc3d49dc9bec5b59; no voters:
I20260811 09:15:41.993616 1084 leader_election.cc:288] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:15:41.993748 1092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:41.994385 1092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [term 1 LEADER]: Becoming Leader. State: Replica: 858eba09fe31466fbc3d49dc9bec5b59, State: Running, Role: LEADER
I20260811 09:15:41.994906 1092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858eba09fe31466fbc3d49dc9bec5b59" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41967 } }
I20260811 09:15:41.995587 1084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:15:41.997551 1094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 858eba09fe31466fbc3d49dc9bec5b59. Latest consensus state: current_term: 1 leader_uuid: "858eba09fe31466fbc3d49dc9bec5b59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858eba09fe31466fbc3d49dc9bec5b59" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41967 } } }
I20260811 09:15:41.997960 1094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:41.998044 1093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "858eba09fe31466fbc3d49dc9bec5b59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858eba09fe31466fbc3d49dc9bec5b59" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41967 } } }
I20260811 09:15:41.998400 1093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:42.000358 1102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:15:42.005496 1102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:15:42.014195 1102 catalog_manager.cc:1347] Generated new cluster ID: 2cd9d5f43bce429b97e17752b1fea3e2
I20260811 09:15:42.014309 1102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:15:42.033915 1102 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:15:42.035116 1102 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:15:42.044513 1102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 858eba09fe31466fbc3d49dc9bec5b59: Generated new TSK 0
I20260811 09:15:42.045537 1102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:15:42.060627 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41967
--builtin_ntp_servers=127.17.16.84:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 09:15:42.391968 1124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:42.392258 1124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:42.392400 1124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:42.399855 1124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:42.399969 1124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:42.400120 1124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:15:42.409224 1124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:42.410596 1124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:42.412425 1124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:42.425935 1134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:42.426074 1135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:42.426076 1140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:42.427035 1124 server_base.cc:1034] running on GCE node
I20260811 09:15:42.427676 1124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:42.428654 1124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:42.429848 1124 hybrid_clock.cc:648] HybridClock initialized: now 1786439742429781 us; error 61 us; skew 500 ppm
I20260811 09:15:42.430161 1124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:15:42.440069 1124 webserver.cc:460] Webserver started at http://127.17.16.65:34887/ using document root <none> and password file <none>
I20260811 09:15:42.440807 1124 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:42.440930 1124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:42.441262 1124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:42.444128 1124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "91fab7e1393742dc96649664729c1279"
format_stamp: "Formatted at 2026-08-11 09:15:42 on dist-test-slave-ncp9"
server_key: "e323a20d967bf4b5e752a61ea6697c96"
server_key_iv: "9d0e50fba246868701a4ccfe20c5c349"
server_key_version: "encryptionkey@0"
I20260811 09:15:42.444928 1124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "91fab7e1393742dc96649664729c1279"
format_stamp: "Formatted at 2026-08-11 09:15:42 on dist-test-slave-ncp9"
server_key: "e323a20d967bf4b5e752a61ea6697c96"
server_key_iv: "9d0e50fba246868701a4ccfe20c5c349"
server_key_version: "encryptionkey@0"
I20260811 09:15:42.449934 1124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:15:42.453305 1149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:42.454509 1124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 09:15:42.454665 1124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "91fab7e1393742dc96649664729c1279"
format_stamp: "Formatted at 2026-08-11 09:15:42 on dist-test-slave-ncp9"
server_key: "e323a20d967bf4b5e752a61ea6697c96"
server_key_iv: "9d0e50fba246868701a4ccfe20c5c349"
server_key_version: "encryptionkey@0"
I20260811 09:15:42.454862 1124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:42.492509 1124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:42.493485 1124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:42.493784 1124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:42.494848 1124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:42.496379 1124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:42.496485 1124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:42.496588 1124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:42.496656 1124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:42.527197 1124 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:44581
I20260811 09:15:42.527242 1334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:44581 every 8 connection(s)
I20260811 09:15:42.528483 1124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:15:42.528829 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 1124
I20260811 09:15:42.529048 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:15:42.529870 17473 external_mini_cluster.cc:1428] Setting key c9098827bc51de9fcd788c348c4356bc
I20260811 09:15:42.532842 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41967
--builtin_ntp_servers=127.17.16.84:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 09:15:42.541038 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:42.531058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:46453 (local address 127.17.16.126:41967)
0811 09:15:42.531480 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:42.531485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:42.533127 (+ 1642us) server_negotiation.cc:407] Connection header received
0811 09:15:42.534805 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:42.534811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:42.534892 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:42.534987 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:42.536037 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:42.536761 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:42.538304 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:42.538712 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:42.538825 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:42.540168 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:42.540172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:42.540175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:42.540194 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:42.540235 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:42.540247 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:42.540320 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:42.540536 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 09:15:42.540643 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":224,"threads_started":1}
I20260811 09:15:42.542089 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:42.531793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41967 (local address 127.17.16.65:46453)
0811 09:15:42.532917 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:42.532969 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:42.534554 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:42.535221 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:42.535230 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:42.535312 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:42.535863 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:42.535880 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:42.536934 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:42.536940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:42.538162 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:42.538174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:42.538931 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:42.538939 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:42.539166 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:42.539853 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:15:42.539856 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:42.539895 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:15:42.539979 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:42.540337 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:42.540348 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:42.540356 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:15:42.540511 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 09:15:42.540957 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":354,"threads_started":1}
I20260811 09:15:42.543704 1335 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41967
I20260811 09:15:42.544047 1335 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:42.545040 1335 heartbeater.cc:511] Master 127.17.16.126:41967 requested a full tablet report, sending...
I20260811 09:15:42.547317 1032 ts_manager.cc:194] Registered new tserver with Master: 91fab7e1393742dc96649664729c1279 (127.17.16.65:44581)
I20260811 09:15:42.549269 1032 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.65:46453
W20260811 09:15:42.868070 1342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:42.868342 1342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:42.868433 1342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:42.875586 1342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:42.875682 1342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:42.875792 1342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:15:42.884585 1342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:42.885782 1342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:42.888000 1342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:42.900795 1356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:42.900758 1353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:42.901104 1352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:42.901697 1342 server_base.cc:1034] running on GCE node
I20260811 09:15:42.902312 1342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:42.903259 1342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:42.904449 1342 hybrid_clock.cc:648] HybridClock initialized: now 1786439742904386 us; error 55 us; skew 500 ppm
I20260811 09:15:42.904762 1342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:15:42.906965 1342 webserver.cc:460] Webserver started at http://127.17.16.66:41483/ using document root <none> and password file <none>
I20260811 09:15:42.907665 1342 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:42.907763 1342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:42.908092 1342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:42.910957 1342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "aa852d6db0a349eb8f530c00100949da"
format_stamp: "Formatted at 2026-08-11 09:15:42 on dist-test-slave-ncp9"
server_key: "b375e4ae79ee974549749be9a824ff82"
server_key_iv: "f750dce25c180ea8953ce5550f39570a"
server_key_version: "encryptionkey@0"
I20260811 09:15:42.911741 1342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "aa852d6db0a349eb8f530c00100949da"
format_stamp: "Formatted at 2026-08-11 09:15:42 on dist-test-slave-ncp9"
server_key: "b375e4ae79ee974549749be9a824ff82"
server_key_iv: "f750dce25c180ea8953ce5550f39570a"
server_key_version: "encryptionkey@0"
I20260811 09:15:42.916761 1342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:15:42.921046 1366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:42.922315 1342 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:15:42.922468 1342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "aa852d6db0a349eb8f530c00100949da"
format_stamp: "Formatted at 2026-08-11 09:15:42 on dist-test-slave-ncp9"
server_key: "b375e4ae79ee974549749be9a824ff82"
server_key_iv: "f750dce25c180ea8953ce5550f39570a"
server_key_version: "encryptionkey@0"
I20260811 09:15:42.922677 1342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:42.939566 1342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:42.940384 1342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:42.940680 1342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:42.941807 1342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:42.943293 1342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:42.943395 1342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:42.943498 1342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:42.943568 1342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:42.974632 1342 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:38447
I20260811 09:15:42.974686 1536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:38447 every 8 connection(s)
I20260811 09:15:42.975834 1342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:15:42.983296 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 1342
I20260811 09:15:42.983502 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:15:42.984058 17473 external_mini_cluster.cc:1428] Setting key 995fce8453c4bd6f635eb1c3820ed5a8
I20260811 09:15:42.987427 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41967
--builtin_ntp_servers=127.17.16.84:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 09:15:42.988493 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:42.978216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:56443 (local address 127.17.16.126:41967)
0811 09:15:42.978387 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:42.978391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:42.979713 (+ 1322us) server_negotiation.cc:407] Connection header received
0811 09:15:42.981466 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:42.981470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:42.981562 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:42.981651 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:42.982722 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:42.983487 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:42.985490 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:42.986015 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:42.986142 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:42.987812 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:42.987817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:42.987820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:42.987840 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:42.987874 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:42.987886 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:42.987956 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:42.988175 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 09:15:42.988259 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:15:42.989965 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:42.978690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41967 (local address 127.17.16.66:56443)
0811 09:15:42.979549 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:42.979591 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:42.981209 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:42.981849 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:42.981857 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:42.981945 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:42.982564 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:42.982583 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:42.983663 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:42.983669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:42.985340 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:42.985360 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:42.986242 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:42.986250 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:42.986548 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:42.987351 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:15:42.987354 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:42.987414 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:15:42.987531 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:42.987974 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:42.987985 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:42.987993 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:15:42.988177 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 09:15:42.988590 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":259,"threads_started":1}
I20260811 09:15:42.991441 1539 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41967
I20260811 09:15:42.991768 1539 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:42.992676 1539 heartbeater.cc:511] Master 127.17.16.126:41967 requested a full tablet report, sending...
I20260811 09:15:42.994232 1032 ts_manager.cc:194] Registered new tserver with Master: aa852d6db0a349eb8f530c00100949da (127.17.16.66:38447)
I20260811 09:15:42.995136 1032 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.66:56443
W20260811 09:15:43.316362 1542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:43.316659 1542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:43.316756 1542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:43.323724 1542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:43.323810 1542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:43.323917 1542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:15:43.332541 1542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:43.333750 1542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:43.335391 1542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:43.347761 1552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:43.347862 1553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:43.347924 1556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:43.348933 1542 server_base.cc:1034] running on GCE node
I20260811 09:15:43.349524 1542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:43.350431 1542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:43.351624 1542 hybrid_clock.cc:648] HybridClock initialized: now 1786439743351581 us; error 35 us; skew 500 ppm
I20260811 09:15:43.351897 1542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:15:43.354002 1542 webserver.cc:460] Webserver started at http://127.17.16.67:36453/ using document root <none> and password file <none>
I20260811 09:15:43.354651 1542 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:43.354734 1542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:43.355008 1542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:43.357829 1542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "bde6309ab13246bfb995e909b7cae6c0"
format_stamp: "Formatted at 2026-08-11 09:15:43 on dist-test-slave-ncp9"
server_key: "e4a5aa5df6325a3c972a3092aceef0cc"
server_key_iv: "6a02c698fc69f52d4741a926df052639"
server_key_version: "encryptionkey@0"
I20260811 09:15:43.358683 1542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "bde6309ab13246bfb995e909b7cae6c0"
format_stamp: "Formatted at 2026-08-11 09:15:43 on dist-test-slave-ncp9"
server_key: "e4a5aa5df6325a3c972a3092aceef0cc"
server_key_iv: "6a02c698fc69f52d4741a926df052639"
server_key_version: "encryptionkey@0"
I20260811 09:15:43.363651 1542 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:15:43.366827 1563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:43.368036 1542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 09:15:43.368167 1542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "bde6309ab13246bfb995e909b7cae6c0"
format_stamp: "Formatted at 2026-08-11 09:15:43 on dist-test-slave-ncp9"
server_key: "e4a5aa5df6325a3c972a3092aceef0cc"
server_key_iv: "6a02c698fc69f52d4741a926df052639"
server_key_version: "encryptionkey@0"
I20260811 09:15:43.368338 1542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:43.378764 1542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:43.379464 1542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:43.379707 1542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:43.380590 1542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:43.381940 1542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:43.382042 1542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:43.382121 1542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:43.382179 1542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:43.413144 1542 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:40979
I20260811 09:15:43.413174 1746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:40979 every 8 connection(s)
I20260811 09:15:43.414521 1542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:15:43.424515 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 1542
I20260811 09:15:43.424710 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:15:43.425299 17473 external_mini_cluster.cc:1428] Setting key ce8f8077dc187016bd001ab886c4dae6
I20260811 09:15:43.427707 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:43.417093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:48289 (local address 127.17.16.126:41967)
0811 09:15:43.417258 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:43.417263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:43.418687 (+ 1424us) server_negotiation.cc:407] Connection header received
0811 09:15:43.420543 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:43.420548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:43.420629 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:43.420730 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:43.421814 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:43.422630 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:43.424219 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:43.424731 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:43.424879 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:43.426840 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:43.426845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:43.426848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:43.426868 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:43.426904 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:43.426917 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:43.427007 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:43.427287 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 09:15:43.427377 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 09:15:43.428850 1751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:43.417662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41967 (local address 127.17.16.67:48289)
0811 09:15:43.418525 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:43.418565 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:43.420234 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:43.420943 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:43.420952 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:43.421049 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:43.421632 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:43.421658 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:43.422764 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:43.422768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:43.424023 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:43.424065 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:43.425047 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:43.425058 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:43.425380 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:43.426412 (+ 1032us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:15:43.426414 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:43.426462 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:15:43.426586 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:43.427024 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:43.427038 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:43.427047 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 09:15:43.427239 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 09:15:43.427631 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":269,"threads_started":1}
I20260811 09:15:43.430105 1749 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41967
I20260811 09:15:43.430419 1749 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:43.431130 1749 heartbeater.cc:511] Master 127.17.16.126:41967 requested a full tablet report, sending...
I20260811 09:15:43.432408 1032 ts_manager.cc:194] Registered new tserver with Master: bde6309ab13246bfb995e909b7cae6c0 (127.17.16.67:40979)
I20260811 09:15:43.433207 1032 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.67:48289
I20260811 09:15:43.440433 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 09:15:43.456759 1339 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:43.454776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45144 (local address 127.17.16.126:41967)
0811 09:15:43.455018 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:43.455025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:43.455472 (+ 447us) server_negotiation.cc:407] Connection header received
0811 09:15:43.455766 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:43.455773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:43.455867 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:43.455965 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:43.456487 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45144: BlockingRecv error: recv got EOF from 127.0.0.1:45144 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 09:15:43.457748 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 1124
I20260811 09:15:43.465667 1761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:43.458639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44784 (local address 127.17.16.65:44581)
0811 09:15:43.459141 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:43.459151 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:43.459188 (+ 37us) server_negotiation.cc:407] Connection header received
0811 09:15:43.459256 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:43.459263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:43.459485 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:43.459633 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:43.459932 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:43.461130 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:43.462442 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:43.463078 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:43.463197 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:43.464212 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:43.464222 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:43.464237 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:43.464872 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:43.464999 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:43.465025 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:43.465121 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:43.465235 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 09:15:43.465364 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":239,"threads_started":1}
I20260811 09:15:43.467092 1329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:43.548312 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 1124
I20260811 09:15:43.572460 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 1342
I20260811 09:15:43.579957 1763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:43.573335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.17.16.66:38447)
0811 09:15:43.573921 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:43.573929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:43.573956 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:15:43.574019 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:43.574026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:43.574249 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:43.574377 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:43.574724 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:43.575730 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:43.576880 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:43.577557 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:43.577666 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:43.578588 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:43.578595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:43.578608 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:43.579204 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:43.579254 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:43.579276 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:43.579364 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:43.579488 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 09:15:43.579621 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":318,"threads_started":1}
I20260811 09:15:43.581250 1527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:43.661003 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 1342
I20260811 09:15:43.683028 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 1542
I20260811 09:15:43.690758 1767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:43.683895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.17.16.67:40979)
0811 09:15:43.684495 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:43.684503 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:43.684533 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:15:43.684595 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:43.684601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:43.684820 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:43.684980 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:43.685369 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:43.686376 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:43.687606 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:43.688158 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:43.688333 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:43.689330 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:43.689338 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:43.689352 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:43.689976 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:43.690025 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:43.690046 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:43.690133 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:43.690273 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 09:15:43.690385 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":296,"threads_started":1}
I20260811 09:15:43.692036 1742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:43.774549 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 1542
I20260811 09:15:43.803478 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 978
I20260811 09:15:43.804679 1076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:43.906509 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 978
2026-08-11T09:15:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2462 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T09:15:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:15:43Z Disabled control of system clock
I20260811 09:15:43.994686 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:45929
--webserver_interface=127.17.16.126
--webserver_port=0
--builtin_ntp_servers=127.17.16.84:35203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:45929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 09:15:44.321069 1774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:44.321431 1774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:44.321568 1774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:44.328761 1774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:15:44.328863 1774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:44.328949 1774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:44.329037 1774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:15:44.329097 1774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:15:44.338131 1774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:35203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:45929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:45929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:44.339360 1774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:44.341164 1774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:44.352506 1781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:44.352506 1783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:44.352546 1787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:44.353217 1774 server_base.cc:1034] running on GCE node
I20260811 09:15:44.353945 1774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:44.355283 1774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:44.356485 1774 hybrid_clock.cc:648] HybridClock initialized: now 1786439744356450 us; error 26 us; skew 500 ppm
I20260811 09:15:44.356824 1774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:15:44.358969 1774 webserver.cc:460] Webserver started at http://127.17.16.126:43431/ using document root <none> and password file <none>
I20260811 09:15:44.359637 1774 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:44.359738 1774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:44.360039 1774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:44.362953 1774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007"
format_stamp: "Formatted at 2026-08-11 09:15:44 on dist-test-slave-ncp9"
server_key: "b0f98b361ea04b6130d3c0e25f2a0062"
server_key_iv: "6a89cb2f2ff8cf2ec7ffbe940a1e1649"
server_key_version: "encryptionkey@0"
I20260811 09:15:44.363715 1774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007"
format_stamp: "Formatted at 2026-08-11 09:15:44 on dist-test-slave-ncp9"
server_key: "b0f98b361ea04b6130d3c0e25f2a0062"
server_key_iv: "6a89cb2f2ff8cf2ec7ffbe940a1e1649"
server_key_version: "encryptionkey@0"
I20260811 09:15:44.368618 1774 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 09:15:44.371845 1799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:44.373122 1774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:44.373278 1774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007"
format_stamp: "Formatted at 2026-08-11 09:15:44 on dist-test-slave-ncp9"
server_key: "b0f98b361ea04b6130d3c0e25f2a0062"
server_key_iv: "6a89cb2f2ff8cf2ec7ffbe940a1e1649"
server_key_version: "encryptionkey@0"
I20260811 09:15:44.373474 1774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:44.378815 1774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:44.379563 1774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:44.379837 1774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:44.397176 1774 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:45929
I20260811 09:15:44.397220 1872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:45929 every 8 connection(s)
I20260811 09:15:44.398747 1774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:15:44.402454 1873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:44.407899 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 1774
I20260811 09:15:44.408084 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:15:44.408710 17473 external_mini_cluster.cc:1428] Setting key 9ad3a11c348a614b1af9eac875002a48
I20260811 09:15:44.412833 1873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007: Bootstrap starting.
I20260811 09:15:44.416277 1873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:44.416495 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:44.410687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51648 (local address 127.17.16.126:45929)
0811 09:15:44.411605 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:44.411615 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:44.411655 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:15:44.413301 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:44.413323 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:44.413700 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:44.413987 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:44.415035 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:44.415042 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:44.415053 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:44.415066 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:44.415125 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:44.415139 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:44.415292 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:44.415383 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 09:15:44.415635 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":313,"threads_started":1}
I20260811 09:15:44.417616 1873 log.cc:824] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:44.420415 1873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007: No bootstrap required, opened a new log
I20260811 09:15:44.427367 1873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45929 } }
I20260811 09:15:44.427677 1873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:44.427752 1873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c6d3fc6a6ac4ebeb04f23c79ebda007, State: Initialized, Role: FOLLOWER
I20260811 09:15:44.428258 1873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45929 } }
I20260811 09:15:44.428458 1873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:15:44.428552 1873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:15:44.428696 1873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:44.430878 1873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45929 } }
I20260811 09:15:44.431306 1873 leader_election.cc:302] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c6d3fc6a6ac4ebeb04f23c79ebda007; no voters:
I20260811 09:15:44.431802 1873 leader_election.cc:288] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:15:44.431907 1880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:44.432598 1880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [term 1 LEADER]: Becoming Leader. State: Replica: 4c6d3fc6a6ac4ebeb04f23c79ebda007, State: Running, Role: LEADER
I20260811 09:15:44.433173 1880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45929 } }
I20260811 09:15:44.433717 1873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:15:44.435830 1883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c6d3fc6a6ac4ebeb04f23c79ebda007. Latest consensus state: current_term: 1 leader_uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45929 } } }
I20260811 09:15:44.436213 1883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:44.436985 1882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c6d3fc6a6ac4ebeb04f23c79ebda007" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45929 } } }
I20260811 09:15:44.437348 1882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:44.437906 1892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:15:44.442665 1892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:15:44.451982 1892 catalog_manager.cc:1347] Generated new cluster ID: d9c43f3162334062ba50671493a4583d
I20260811 09:15:44.452075 1892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:15:44.493655 1892 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:15:44.494083 1892 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:15:44.514853 1892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c6d3fc6a6ac4ebeb04f23c79ebda007: Generated new TSK 0
I20260811 09:15:44.515725 1892 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:15:44.528301 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45929
--builtin_ntp_servers=127.17.16.84:35203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 09:15:44.888800 1910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:44.889149 1910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:44.889281 1910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:44.896598 1910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:44.896693 1910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:44.896835 1910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:15:44.905879 1910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:35203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:44.907176 1910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:44.909039 1910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:44.921770 1916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:44.921708 1919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:44.921747 1917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:44.922967 1910 server_base.cc:1034] running on GCE node
I20260811 09:15:44.923590 1910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:44.924573 1910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:44.925782 1910 hybrid_clock.cc:648] HybridClock initialized: now 1786439744925734 us; error 35 us; skew 500 ppm
I20260811 09:15:44.926110 1910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:15:44.928324 1910 webserver.cc:460] Webserver started at http://127.17.16.65:38589/ using document root <none> and password file <none>
I20260811 09:15:44.929056 1910 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:44.929163 1910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:44.929482 1910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:44.932310 1910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "42b146a2b1894faf9b5ed852c4dcc038"
format_stamp: "Formatted at 2026-08-11 09:15:44 on dist-test-slave-ncp9"
server_key: "150ccd3c72ae354c97aa09d4710e1980"
server_key_iv: "4c08bb1c4c5629e73fd5c9fd66bc3909"
server_key_version: "encryptionkey@0"
I20260811 09:15:44.933151 1910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "42b146a2b1894faf9b5ed852c4dcc038"
format_stamp: "Formatted at 2026-08-11 09:15:44 on dist-test-slave-ncp9"
server_key: "150ccd3c72ae354c97aa09d4710e1980"
server_key_iv: "4c08bb1c4c5629e73fd5c9fd66bc3909"
server_key_version: "encryptionkey@0"
I20260811 09:15:44.938687 1910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:15:44.942134 1932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:44.943415 1910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:15:44.943595 1910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "42b146a2b1894faf9b5ed852c4dcc038"
format_stamp: "Formatted at 2026-08-11 09:15:44 on dist-test-slave-ncp9"
server_key: "150ccd3c72ae354c97aa09d4710e1980"
server_key_iv: "4c08bb1c4c5629e73fd5c9fd66bc3909"
server_key_version: "encryptionkey@0"
I20260811 09:15:44.943797 1910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:44.949473 1910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:44.950222 1910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:44.950459 1910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:44.951339 1910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:44.952687 1910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:44.952781 1910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:44.952886 1910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:44.953011 1910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 09:15:44.982489 1910 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:40789
I20260811 09:15:44.982579 2111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:40789 every 8 connection(s)
I20260811 09:15:44.983805 1910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:15:44.991628 2117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:44.986306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:49499 (local address 127.17.16.126:45929)
0811 09:15:44.986747 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:44.986752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:44.987932 (+ 1180us) server_negotiation.cc:407] Connection header received
0811 09:15:44.989630 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:44.989635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:44.989698 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:44.989785 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:44.990967 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:44.990972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:44.990976 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:44.990991 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:44.991037 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:44.991049 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:44.991142 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:44.991334 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 09:15:44.991387 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":210,"threads_started":1}
I20260811 09:15:44.992420 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:44.986867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:45929 (local address 127.17.16.65:49499)
0811 09:15:44.987777 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:44.987823 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:44.989376 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:44.989970 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:44.989978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:44.990061 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:44.990685 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:15:44.990687 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:44.990724 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:15:44.990809 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:44.991143 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:44.991152 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:44.991160 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:15:44.991380 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 09:15:44.991560 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":270,"threads_started":1}
I20260811 09:15:44.993058 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 1910
I20260811 09:15:44.993227 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:15:44.993841 17473 external_mini_cluster.cc:1428] Setting key 3f26e71658841f66bd8023fe5b2433aa
I20260811 09:15:44.993888 2112 heartbeater.cc:347] Connected to a master server at 127.17.16.126:45929
I20260811 09:15:44.994163 2112 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:44.994805 2112 heartbeater.cc:511] Master 127.17.16.126:45929 requested a full tablet report, sending...
I20260811 09:15:44.996640 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45929
--builtin_ntp_servers=127.17.16.84:35203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 09:15:44.996831 1820 ts_manager.cc:194] Registered new tserver with Master: 42b146a2b1894faf9b5ed852c4dcc038 (127.17.16.65:40789)
W20260811 09:15:45.323313 2119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:45.323585 2119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:45.323675 2119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:45.330950 2119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:45.331072 2119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:45.331198 2119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:15:45.340387 2119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:35203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:45.341683 2119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:45.343577 2119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:45.356117 2124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:45.356211 2125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:45.356551 2129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:45.357132 2119 server_base.cc:1034] running on GCE node
I20260811 09:15:45.357702 2119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:45.358573 2119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:45.359797 2119 hybrid_clock.cc:648] HybridClock initialized: now 1786439745359736 us; error 46 us; skew 500 ppm
I20260811 09:15:45.360124 2119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:15:45.362367 2119 webserver.cc:460] Webserver started at http://127.17.16.66:43361/ using document root <none> and password file <none>
I20260811 09:15:45.363039 2119 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:45.363142 2119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:45.363443 2119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:45.366254 2119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "7cf5c1bc00f945299d30f44d354e50dd"
format_stamp: "Formatted at 2026-08-11 09:15:45 on dist-test-slave-ncp9"
server_key: "313df2fcb4c25d63e734f97d6576656e"
server_key_iv: "a52a9bd4cde843b7023b2f6edbe73662"
server_key_version: "encryptionkey@0"
I20260811 09:15:45.367038 2119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "7cf5c1bc00f945299d30f44d354e50dd"
format_stamp: "Formatted at 2026-08-11 09:15:45 on dist-test-slave-ncp9"
server_key: "313df2fcb4c25d63e734f97d6576656e"
server_key_iv: "a52a9bd4cde843b7023b2f6edbe73662"
server_key_version: "encryptionkey@0"
I20260811 09:15:45.372365 2119 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:15:45.375869 2137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:45.377236 2119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:45.377385 2119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "7cf5c1bc00f945299d30f44d354e50dd"
format_stamp: "Formatted at 2026-08-11 09:15:45 on dist-test-slave-ncp9"
server_key: "313df2fcb4c25d63e734f97d6576656e"
server_key_iv: "a52a9bd4cde843b7023b2f6edbe73662"
server_key_version: "encryptionkey@0"
I20260811 09:15:45.377606 2119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:45.383047 2119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:45.383803 2119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:45.384050 2119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:45.384976 2119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:45.386298 2119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:45.386394 2119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:45.386495 2119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:45.386569 2119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:45.417919 2119 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:41325
I20260811 09:15:45.417969 2313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:41325 every 8 connection(s)
I20260811 09:15:45.419190 2119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:15:45.423312 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 2119
I20260811 09:15:45.423452 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:15:45.423897 17473 external_mini_cluster.cc:1428] Setting key 1b17d8d69ee87749cd1ed3574f5c4f44
I20260811 09:15:45.426609 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45929
--builtin_ntp_servers=127.17.16.84:35203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 09:15:45.427222 2117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:45.421585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:42247 (local address 127.17.16.126:45929)
0811 09:15:45.421747 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:45.421752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:45.423109 (+ 1357us) server_negotiation.cc:407] Connection header received
0811 09:15:45.424964 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:45.424970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:45.425038 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:45.425126 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:45.426455 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:45.426460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:45.426463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:45.426478 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:45.426509 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:45.426521 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:45.426631 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:45.426881 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 09:15:45.426949 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:15:45.428078 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:45.422065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:45929 (local address 127.17.16.66:42247)
0811 09:15:45.422953 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:45.422989 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:45.424682 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:45.425358 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:45.425366 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:45.425450 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:45.426159 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:15:45.426162 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:45.426198 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:15:45.426281 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:45.426612 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:45.426623 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:45.426629 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 09:15:45.426941 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 09:15:45.427176 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":249,"threads_started":1}
I20260811 09:15:45.429386 2316 heartbeater.cc:347] Connected to a master server at 127.17.16.126:45929
I20260811 09:15:45.429659 2316 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:45.430219 2316 heartbeater.cc:511] Master 127.17.16.126:45929 requested a full tablet report, sending...
I20260811 09:15:45.431588 1820 ts_manager.cc:194] Registered new tserver with Master: 7cf5c1bc00f945299d30f44d354e50dd (127.17.16.66:41325)
W20260811 09:15:45.762217 2320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:45.762460 2320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:45.762588 2320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:45.769856 2320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:45.769963 2320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:45.770107 2320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:15:45.779249 2320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:35203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:45929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:45.780525 2320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:45.782473 2320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:45.795066 2331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:45.795004 2333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:45.795043 2337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:45.795717 2320 server_base.cc:1034] running on GCE node
I20260811 09:15:45.796307 2320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:45.797194 2320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:45.798450 2320 hybrid_clock.cc:648] HybridClock initialized: now 1786439745798399 us; error 40 us; skew 500 ppm
I20260811 09:15:45.798789 2320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:15:45.801142 2320 webserver.cc:460] Webserver started at http://127.17.16.67:38891/ using document root <none> and password file <none>
I20260811 09:15:45.801832 2320 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:45.801947 2320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:45.802310 2320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:45.805370 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "07eac10d99f9444db59ad6c586e2bf6c"
format_stamp: "Formatted at 2026-08-11 09:15:45 on dist-test-slave-ncp9"
server_key: "2af7ba93b4a25db3973279d0882b6e04"
server_key_iv: "c0f1afa84168960e554eb905a3efe86b"
server_key_version: "encryptionkey@0"
I20260811 09:15:45.806248 2320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "07eac10d99f9444db59ad6c586e2bf6c"
format_stamp: "Formatted at 2026-08-11 09:15:45 on dist-test-slave-ncp9"
server_key: "2af7ba93b4a25db3973279d0882b6e04"
server_key_iv: "c0f1afa84168960e554eb905a3efe86b"
server_key_version: "encryptionkey@0"
I20260811 09:15:45.811941 2320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 09:15:45.815599 2352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:45.817024 2320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:15:45.817169 2320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "07eac10d99f9444db59ad6c586e2bf6c"
format_stamp: "Formatted at 2026-08-11 09:15:45 on dist-test-slave-ncp9"
server_key: "2af7ba93b4a25db3973279d0882b6e04"
server_key_iv: "c0f1afa84168960e554eb905a3efe86b"
server_key_version: "encryptionkey@0"
I20260811 09:15:45.817379 2320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:45.822736 2320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:45.823477 2320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:45.823722 2320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:45.824589 2320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:45.826005 2320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:45.826092 2320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:45.826200 2320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:45.826267 2320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:45.855407 2320 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:33715
I20260811 09:15:45.855458 2548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:33715 every 8 connection(s)
I20260811 09:15:45.856691 2320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:15:45.864480 2117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:45.859198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:50207 (local address 127.17.16.126:45929)
0811 09:15:45.859370 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:45.859375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:45.860750 (+ 1375us) server_negotiation.cc:407] Connection header received
0811 09:15:45.862444 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:45.862450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:45.862529 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:45.862620 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:45.863799 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:45.863805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:45.863810 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:45.863828 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:45.863861 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:45.863873 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:45.863983 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:45.864154 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 09:15:45.864219 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:15:45.865346 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:45.859705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:45929 (local address 127.17.16.67:50207)
0811 09:15:45.860601 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:45.860644 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:45.862168 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:45.862844 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:45.862852 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:45.862936 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:45.863497 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:15:45.863500 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:45.863556 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:15:45.863638 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:45.863958 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:45.863969 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:45.863976 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 09:15:45.864222 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 09:15:45.864450 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":262,"threads_started":1}
I20260811 09:15:45.866575 2549 heartbeater.cc:347] Connected to a master server at 127.17.16.126:45929
I20260811 09:15:45.866761 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 2320
I20260811 09:15:45.866848 2549 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:45.866911 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:15:45.867345 17473 external_mini_cluster.cc:1428] Setting key 00dd90b99e887799bd1853faa201442e
I20260811 09:15:45.867412 2549 heartbeater.cc:511] Master 127.17.16.126:45929 requested a full tablet report, sending...
I20260811 09:15:45.868789 1820 ts_manager.cc:194] Registered new tserver with Master: 07eac10d99f9444db59ad6c586e2bf6c (127.17.16.67:33715)
I20260811 09:15:45.870241 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 09:15:45.875874 2117 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:15:45.874453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51658 (local address 127.17.16.126:45929)
0811 09:15:45.874613 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:45.874619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:45.875147 (+ 528us) server_negotiation.cc:407] Connection header received
0811 09:15:45.875260 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:45.875265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:45.875324 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:45.875402 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:45.875654 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51658: BlockingRecv error: recv got EOF from 127.0.0.1:51658 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 09:15:45.876441 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 1910
I20260811 09:15:45.879788 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:45.877344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.17.16.65:40789)
0811 09:15:45.877848 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:45.877854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:45.877877 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:15:45.877930 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:45.877936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:45.878095 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:45.878226 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:45.879101 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:45.879108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:45.879118 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:45.879138 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:45.879177 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:45.879191 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:45.879308 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:45.879431 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 09:15:45.879526 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":231,"threads_started":1}
I20260811 09:15:45.880882 2106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:45.960743 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 1910
I20260811 09:15:45.983788 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 2119
I20260811 09:15:45.987313 2570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:45.984662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59174 (local address 127.17.16.66:41325)
0811 09:15:45.985169 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:45.985176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:45.985204 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:15:45.985270 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:45.985276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:45.985490 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:45.985634 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:45.986585 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:45.986592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:45.986603 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:45.986621 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:45.986673 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:45.986689 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:45.986809 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:45.986949 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 09:15:45.987057 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":239,"threads_started":1}
I20260811 09:15:45.988467 2306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:46.067884 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 2119
I20260811 09:15:46.088151 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 2320
I20260811 09:15:46.091902 2572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:46.089173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36630 (local address 127.17.16.67:33715)
0811 09:15:46.089796 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:46.089804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:46.089835 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:15:46.089901 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:46.089907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:46.090125 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:46.090290 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:46.091226 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:46.091233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:46.091245 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:15:46.091261 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:46.091302 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:15:46.091316 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:46.091444 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:46.091496 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 09:15:46.091575 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":310,"threads_started":1}
I20260811 09:15:46.093228 2536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:46.173730 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 2320
I20260811 09:15:46.196115 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 1774
I20260811 09:15:46.197515 1867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:46.299214 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 1774
2026-08-11T09:15:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2392 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:15:46 dist-test-slave-ncp9 krb5kdc[2581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:15:46 dist-test-slave-ncp9 krb5kdc[2581](info): set up 2 sockets
Aug 11 09:15:46 dist-test-slave-ncp9 krb5kdc[2581](info): commencing operation
krb5kdc: starting...
W20260811 09:15:47.541785 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:15:47 dist-test-slave-ncp9 krb5kdc[2581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:15:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:15:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:15:47.805552 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:37157
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:34229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:37157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:48.132043 2602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:48.132323 2602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:48.132428 2602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:48.139379 2602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:15:48.139479 2602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:48.139534 2602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:48.139590 2602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:15:48.139631 2602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:15:48.148917 2602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:34229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:37157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:37157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:48.150094 2602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:48.151305 2602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:48.162475 2617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:48.162516 2612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:48.162417 2610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:48.163071 2602 server_base.cc:1034] running on GCE node
I20260811 09:15:48.163786 2602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:48.165040 2602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:48.166260 2602 hybrid_clock.cc:648] HybridClock initialized: now 1786439748166217 us; error 28 us; skew 500 ppm
I20260811 09:15:48.166615 2602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:48 dist-test-slave-ncp9 krb5kdc[2581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439748, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:48.173024 2602 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:15:48.174448 2602 webserver.cc:460] Webserver started at http://127.17.16.126:42381/ using document root <none> and password file <none>
I20260811 09:15:48.175096 2602 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:48.175206 2602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:48.175495 2602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:48.178453 2602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "32a0857efd9c4f909800ea704e449d86"
format_stamp: "Formatted at 2026-08-11 09:15:48 on dist-test-slave-ncp9"
server_key: "8ac07e806b1d179b9b9d9d3b3db88cc5"
server_key_iv: "0e18088d33a0ac0a65e4e7e414b7700d"
server_key_version: "encryptionkey@0"
I20260811 09:15:48.179291 2602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "32a0857efd9c4f909800ea704e449d86"
format_stamp: "Formatted at 2026-08-11 09:15:48 on dist-test-slave-ncp9"
server_key: "8ac07e806b1d179b9b9d9d3b3db88cc5"
server_key_iv: "0e18088d33a0ac0a65e4e7e414b7700d"
server_key_version: "encryptionkey@0"
I20260811 09:15:48.184428 2602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:15:48.187729 2632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:48.189060 2602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:15:48.189220 2602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "32a0857efd9c4f909800ea704e449d86"
format_stamp: "Formatted at 2026-08-11 09:15:48 on dist-test-slave-ncp9"
server_key: "8ac07e806b1d179b9b9d9d3b3db88cc5"
server_key_iv: "0e18088d33a0ac0a65e4e7e414b7700d"
server_key_version: "encryptionkey@0"
I20260811 09:15:48.189401 2602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:48.206419 2602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:48.210525 2602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:48.210793 2602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:48.228595 2602 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:37157
I20260811 09:15:48.228627 2709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:37157 every 8 connection(s)
I20260811 09:15:48.229974 2602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:15:48.233486 2710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:48.240017 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 2602
I20260811 09:15:48.240248 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:15:48.241055 17473 external_mini_cluster.cc:1428] Setting key a0ea54aa41373db1b1b7b7111792a6ef
I20260811 09:15:48.244271 2710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86: Bootstrap starting.
I20260811 09:15:48.247303 2710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:48.248307 2710 log.cc:824] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:48.250988 2710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86: No bootstrap required, opened a new log
Aug 11 09:15:48 dist-test-slave-ncp9 krb5kdc[2581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:15:48.258801 2710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a0857efd9c4f909800ea704e449d86" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37157 } }
I20260811 09:15:48.259230 2710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:48.259334 2710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a0857efd9c4f909800ea704e449d86, State: Initialized, Role: FOLLOWER
I20260811 09:15:48.259967 2710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a0857efd9c4f909800ea704e449d86" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37157 } }
I20260811 09:15:48.260214 2710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:15:48.260350 2710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:15:48.260514 2710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:48.262753 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:48.243095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37368 (local address 127.17.16.126:37157)
0811 09:15:48.243823 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:48.243835 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:48.243874 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:15:48.245650 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:48.245676 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:48.246060 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:48.246412 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:48.247937 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:48.249343 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:48.250618 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:48.251309 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:48.251492 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:48.255626 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:48.255640 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:48.255654 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:48.255693 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:48.258639 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:48.259409 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:48.259416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:48.259423 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:48.259594 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:48.260004 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:48.260007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:48.260009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:48.260602 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:48.260959 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:48.261416 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 09:15:48.261806 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":319,"threads_started":1}
I20260811 09:15:48.262952 2710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a0857efd9c4f909800ea704e449d86" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37157 } }
I20260811 09:15:48.263425 2710 leader_election.cc:302] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32a0857efd9c4f909800ea704e449d86; no voters:
I20260811 09:15:48.264045 2710 leader_election.cc:288] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:15:48.264128 2716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:48.265255 2716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [term 1 LEADER]: Becoming Leader. State: Replica: 32a0857efd9c4f909800ea704e449d86, State: Running, Role: LEADER
I20260811 09:15:48.265847 2716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a0857efd9c4f909800ea704e449d86" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37157 } }
I20260811 09:15:48.266163 2710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:15:48.268337 2717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32a0857efd9c4f909800ea704e449d86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a0857efd9c4f909800ea704e449d86" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37157 } } }
I20260811 09:15:48.268644 2717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:48.269443 2718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32a0857efd9c4f909800ea704e449d86. Latest consensus state: current_term: 1 leader_uuid: "32a0857efd9c4f909800ea704e449d86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32a0857efd9c4f909800ea704e449d86" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37157 } } }
I20260811 09:15:48.269793 2718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86 [sys.catalog]: This master's current role is: LEADER
I20260811 09:15:48.270152 2722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:15:48.275171 2722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:15:48.282464 2722 catalog_manager.cc:1347] Generated new cluster ID: 7b6058270830435798dc09b99d8cd551
I20260811 09:15:48.282574 2722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:15:48.329686 2722 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:15:48.331552 2722 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:15:48.338961 2722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86: Generated new TSK 0
I20260811 09:15:48.339744 2722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:15:48.422451 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37157
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:34229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:48.749423 2754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:48.749773 2754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:48.749935 2754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:48.757229 2754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:48.757342 2754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:48.757475 2754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:15:48.767355 2754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:34229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:48.768687 2754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:48.770284 2754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:48.783521 2764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:48.783576 2766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:48.783797 2763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:48.784336 2754 server_base.cc:1034] running on GCE node
I20260811 09:15:48.784989 2754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:48.785959 2754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:48.787184 2754 hybrid_clock.cc:648] HybridClock initialized: now 1786439748787127 us; error 39 us; skew 500 ppm
I20260811 09:15:48.787508 2754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:48 dist-test-slave-ncp9 krb5kdc[2581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439748, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:48.790612 2754 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:15:48.791993 2754 webserver.cc:460] Webserver started at http://127.17.16.65:46371/ using document root <none> and password file <none>
I20260811 09:15:48.792776 2754 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:48.792948 2754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:48.793279 2754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:48.796438 2754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "06cf4c45f7fe4519adc05ef37c29db92"
format_stamp: "Formatted at 2026-08-11 09:15:48 on dist-test-slave-ncp9"
server_key: "2956c2b2032bea1b09928fcd4ad60128"
server_key_iv: "b2b60b6b5f71a8b86c0ee674334ca17a"
server_key_version: "encryptionkey@0"
I20260811 09:15:48.797266 2754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "06cf4c45f7fe4519adc05ef37c29db92"
format_stamp: "Formatted at 2026-08-11 09:15:48 on dist-test-slave-ncp9"
server_key: "2956c2b2032bea1b09928fcd4ad60128"
server_key_iv: "b2b60b6b5f71a8b86c0ee674334ca17a"
server_key_version: "encryptionkey@0"
I20260811 09:15:48.802731 2754 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:15:48.806229 2776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:48.807477 2754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:15:48.807639 2754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "06cf4c45f7fe4519adc05ef37c29db92"
format_stamp: "Formatted at 2026-08-11 09:15:48 on dist-test-slave-ncp9"
server_key: "2956c2b2032bea1b09928fcd4ad60128"
server_key_iv: "b2b60b6b5f71a8b86c0ee674334ca17a"
server_key_version: "encryptionkey@0"
I20260811 09:15:48.807833 2754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:48.838757 2754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:48.842907 2754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:48.843201 2754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:48.844162 2754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:48.845556 2754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:48.845638 2754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:48.845757 2754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:48.845818 2754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:48.876161 2754 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:41261
I20260811 09:15:48.876207 2958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:41261 every 8 connection(s)
I20260811 09:15:48.877485 2754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:15:48.887935 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 2754
I20260811 09:15:48.888126 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:15:48.888767 17473 external_mini_cluster.cc:1428] Setting key 037ce8982901c03123b8a5e760fc2b02
Aug 11 09:15:48 dist-test-slave-ncp9 krb5kdc[2581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439748, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:15:48.896468 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:48.879889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:58935 (local address 127.17.16.126:37157)
0811 09:15:48.880346 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:48.880352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:48.881336 (+ 984us) server_negotiation.cc:407] Connection header received
0811 09:15:48.883058 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:48.883062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:48.883150 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:48.883246 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:48.884562 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:48.885312 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:48.886742 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:48.887133 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:48.887235 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:48.890933 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:48.890939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:48.890942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:48.890974 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:48.893512 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:48.894245 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:48.894252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:48.894254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:48.894373 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:48.894791 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:48.894797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:48.894800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:48.895052 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:48.895191 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:48.895879 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 09:15:48.896099 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":204,"threads_started":1}
I20260811 09:15:48.897396 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:48.880460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37157 (local address 127.17.16.65:58935)
0811 09:15:48.881187 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:48.881226 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:48.882824 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:48.883430 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:48.883438 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:48.883908 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:48.884431 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:48.884448 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:48.885443 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:48.885447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:48.886632 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:48.886643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:48.887305 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:48.887309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:48.887580 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:48.888267 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:48.888290 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:48.890708 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:48.893650 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:48.893660 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:48.893675 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:48.894133 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:48.894478 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:48.894483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:48.894487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:48.894684 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:48.895177 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:48.895185 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:48.895514 (+ 329us) client_negotiation.cc:769] Sending connection context
0811 09:15:48.895763 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 09:15:48.896268 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":281,"threads_started":1}
I20260811 09:15:48.898867 2961 heartbeater.cc:347] Connected to a master server at 127.17.16.126:37157
I20260811 09:15:48.899147 2961 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:48.899830 2961 heartbeater.cc:511] Master 127.17.16.126:37157 requested a full tablet report, sending...
I20260811 09:15:48.901783 2653 ts_manager.cc:194] Registered new tserver with Master: 06cf4c45f7fe4519adc05ef37c29db92 (127.17.16.65:41261)
I20260811 09:15:48.903195 2653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:58935
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:15:48.946302 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37157
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:34229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:49.282498 2974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:49.282763 2974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:49.282907 2974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:49.289950 2974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:49.290114 2974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:49.290277 2974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:15:49.300005 2974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:34229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:49.301267 2974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:49.302537 2974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:49.315091 2981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:49.315158 2982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:49.315271 2985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:49.316330 2974 server_base.cc:1034] running on GCE node
I20260811 09:15:49.316994 2974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:49.317912 2974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:49.319114 2974 hybrid_clock.cc:648] HybridClock initialized: now 1786439749319056 us; error 46 us; skew 500 ppm
I20260811 09:15:49.319412 2974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:49 dist-test-slave-ncp9 krb5kdc[2581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439749, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:49.322507 2974 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:15:49.323851 2974 webserver.cc:460] Webserver started at http://127.17.16.66:41183/ using document root <none> and password file <none>
I20260811 09:15:49.324501 2974 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:49.324605 2974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:49.324868 2974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:49.327764 2974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "0cd19073155b4887a7001a7ab56baf97"
format_stamp: "Formatted at 2026-08-11 09:15:49 on dist-test-slave-ncp9"
server_key: "18ad934765bc473af40af6d242d11023"
server_key_iv: "72f3c38c5303491f2cbd2985862aa721"
server_key_version: "encryptionkey@0"
I20260811 09:15:49.328478 2974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "0cd19073155b4887a7001a7ab56baf97"
format_stamp: "Formatted at 2026-08-11 09:15:49 on dist-test-slave-ncp9"
server_key: "18ad934765bc473af40af6d242d11023"
server_key_iv: "72f3c38c5303491f2cbd2985862aa721"
server_key_version: "encryptionkey@0"
I20260811 09:15:49.333523 2974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:15:49.336795 3000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:49.338096 2974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:15:49.338230 2974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "0cd19073155b4887a7001a7ab56baf97"
format_stamp: "Formatted at 2026-08-11 09:15:49 on dist-test-slave-ncp9"
server_key: "18ad934765bc473af40af6d242d11023"
server_key_iv: "72f3c38c5303491f2cbd2985862aa721"
server_key_version: "encryptionkey@0"
I20260811 09:15:49.338387 2974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:49.351948 2974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:49.355593 2974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:49.355839 2974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:49.356724 2974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:49.358021 2974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:49.358094 2974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:49.358182 2974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:49.358228 2974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:49.388598 2974 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:38029
I20260811 09:15:49.388633 3157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:38029 every 8 connection(s)
I20260811 09:15:49.389873 2974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:15:49.391835 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 2974
I20260811 09:15:49.392011 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:15:49.392517 17473 external_mini_cluster.cc:1428] Setting key 3287b96d4f966d10de20dcf868fb3a09
Aug 11 09:15:49 dist-test-slave-ncp9 krb5kdc[2581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439749, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:15:49.411655 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.392431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:39209 (local address 127.17.16.126:37157)
0811 09:15:49.392635 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:49.392650 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:49.394251 (+ 1601us) server_negotiation.cc:407] Connection header received
0811 09:15:49.396604 (+ 2353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:49.396608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:49.396680 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:49.396753 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:49.398691 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.399501 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.401556 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.401952 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.402040 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.405924 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:49.405929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:49.405932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:49.405964 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:49.408446 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:49.409309 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:49.409313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:49.409315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:49.409442 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:49.409889 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:49.409892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:49.409894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:49.410152 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:49.410268 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:49.411178 (+ 910us) server_negotiation.cc:299] Negotiation successful
0811 09:15:49.411355 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:15:49.412873 3162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.393002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37157 (local address 127.17.16.66:39209)
0811 09:15:49.394109 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:49.394155 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:49.396292 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:49.397035 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:49.397047 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:49.397759 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:49.398504 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:49.398531 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.399654 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.399659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:49.401412 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:49.401426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.402164 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.402171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:49.402451 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.403128 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:49.403155 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:49.405695 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:49.408603 (+ 2908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:49.408612 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:49.408628 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:49.409203 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:49.409554 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:49.409559 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:49.409563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:49.409799 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:49.410274 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:49.410282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:49.410729 (+ 447us) client_negotiation.cc:769] Sending connection context
0811 09:15:49.411004 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 09:15:49.411506 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":630,"threads_started":1}
I20260811 09:15:49.414361 3159 heartbeater.cc:347] Connected to a master server at 127.17.16.126:37157
I20260811 09:15:49.414677 3159 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:49.415421 3159 heartbeater.cc:511] Master 127.17.16.126:37157 requested a full tablet report, sending...
I20260811 09:15:49.416973 2653 ts_manager.cc:194] Registered new tserver with Master: 0cd19073155b4887a7001a7ab56baf97 (127.17.16.66:38029)
I20260811 09:15:49.417788 2653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:39209
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:15:49.455259 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37157
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:34229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:15:49.785935 3165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:15:49.786211 3165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:15:49.786350 3165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:15:49.793507 3165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:15:49.793613 3165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:15:49.793752 3165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:15:49.803066 3165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:34229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:15:49.804271 3165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:15:49.805576 3165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:15:49.818053 3180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:49.818045 3174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:15:49.818626 3173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:15:49.818761 3165 server_base.cc:1034] running on GCE node
I20260811 09:15:49.819326 3165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:15:49.820153 3165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:15:49.821342 3165 hybrid_clock.cc:648] HybridClock initialized: now 1786439749821290 us; error 43 us; skew 500 ppm
I20260811 09:15:49.821631 3165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:15:49 dist-test-slave-ncp9 krb5kdc[2581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439749, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:15:49.824506 3165 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:15:49.825827 3165 webserver.cc:460] Webserver started at http://127.17.16.67:40983/ using document root <none> and password file <none>
I20260811 09:15:49.826478 3165 fs_manager.cc:359] Metadata directory not provided
I20260811 09:15:49.826591 3165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:15:49.826866 3165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:15:49.829699 3165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "07af50e086ea4ffe9eaf20b846f7218e"
format_stamp: "Formatted at 2026-08-11 09:15:49 on dist-test-slave-ncp9"
server_key: "c437c4ed8dea380d830992bb7768ac9b"
server_key_iv: "34f6fbea49b30f71689077c50a900222"
server_key_version: "encryptionkey@0"
I20260811 09:15:49.830400 3165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "07af50e086ea4ffe9eaf20b846f7218e"
format_stamp: "Formatted at 2026-08-11 09:15:49 on dist-test-slave-ncp9"
server_key: "c437c4ed8dea380d830992bb7768ac9b"
server_key_iv: "34f6fbea49b30f71689077c50a900222"
server_key_version: "encryptionkey@0"
I20260811 09:15:49.835469 3165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:15:49.838701 3193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:49.840000 3165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:15:49.840180 3165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "07af50e086ea4ffe9eaf20b846f7218e"
format_stamp: "Formatted at 2026-08-11 09:15:49 on dist-test-slave-ncp9"
server_key: "c437c4ed8dea380d830992bb7768ac9b"
server_key_iv: "34f6fbea49b30f71689077c50a900222"
server_key_version: "encryptionkey@0"
I20260811 09:15:49.840375 3165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:15:49.860678 3165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:15:49.864459 3165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:15:49.864718 3165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:15:49.865664 3165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:15:49.867000 3165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:15:49.867087 3165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:49.867208 3165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:15:49.867259 3165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:15:49.896620 3165 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:40149
I20260811 09:15:49.896667 3373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:40149 every 8 connection(s)
I20260811 09:15:49.897857 3165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:15:49.901553 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 3165
I20260811 09:15:49.901739 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:15:49.902171 17473 external_mini_cluster.cc:1428] Setting key ee1deec7a7c01227a923b8915d4286b1
I20260811 09:15:49.905931 2961 heartbeater.cc:503] Master 127.17.16.126:37157 was elected leader, sending a full tablet report...
Aug 11 09:15:49 dist-test-slave-ncp9 krb5kdc[2581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439749, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:15:49.916770 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.900273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:48349 (local address 127.17.16.126:37157)
0811 09:15:49.900462 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:49.900467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:49.901574 (+ 1107us) server_negotiation.cc:407] Connection header received
0811 09:15:49.903549 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:49.903555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:49.903662 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:49.903746 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:49.905329 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.906121 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.907982 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.908361 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.908455 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.911589 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:49.911593 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:49.911597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:49.911627 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:49.913985 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:49.914635 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:49.914640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:49.914642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:49.914770 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:49.915104 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:49.915107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:49.915109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:49.915359 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:49.915467 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:49.916250 (+ 783us) server_negotiation.cc:299] Negotiation successful
0811 09:15:49.916432 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:15:49.917678 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.900669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37157 (local address 127.17.16.67:48349)
0811 09:15:49.901389 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:49.901426 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:49.903275 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:49.904013 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:49.904024 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:49.904541 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:15:49.905176 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:49.905195 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.906258 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.906262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:49.907836 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:49.907849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.908525 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.908529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:49.908725 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.909332 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:15:49.909359 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:15:49.911401 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:15:49.914112 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:49.914119 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:49.914130 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:49.914509 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:49.914856 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:15:49.914858 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:15:49.914861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:15:49.915024 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:15:49.915484 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:15:49.915492 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:15:49.915813 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 09:15:49.916110 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 09:15:49.916534 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":288,"threads_started":1}
I20260811 09:15:49.918900 3374 heartbeater.cc:347] Connected to a master server at 127.17.16.126:37157
I20260811 09:15:49.919165 3374 heartbeater.cc:464] Registering TS with master...
I20260811 09:15:49.919831 3374 heartbeater.cc:511] Master 127.17.16.126:37157 requested a full tablet report, sending...
I20260811 09:15:49.921097 2653 ts_manager.cc:194] Registered new tserver with Master: 07af50e086ea4ffe9eaf20b846f7218e (127.17.16.67:40149)
I20260811 09:15:49.921859 2653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:48349
I20260811 09:15:49.929189 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:15:49.945039 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.933410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40968 (local address 127.17.16.126:37157)
0811 09:15:49.933562 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:49.933567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:49.934021 (+ 454us) server_negotiation.cc:407] Connection header received
0811 09:15:49.934158 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:49.934162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:49.934221 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:49.934295 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:49.935584 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.936316 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.937549 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.937934 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.938023 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.940066 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:49.940070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:49.940074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:49.940102 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:49.942599 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:49.943267 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:49.943271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:49.943273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:49.943419 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:49.943726 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:49.943729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:49.943731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:49.944030 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:49.944209 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:49.944509 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 09:15:49.944697 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:15:49.948601 2653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:15:49.955276 2653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:15:49.986018 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.968746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40214 (local address 127.17.16.65:41261)
0811 09:15:49.969367 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:49.969376 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:49.978207 (+ 8831us) server_negotiation.cc:407] Connection header received
0811 09:15:49.978505 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:49.978513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:49.978757 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:49.978915 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:49.979761 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.980938 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.983203 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.984639 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.984749 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.985007 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:49.985513 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 09:15:49.985695 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":302,"threads_started":1}
I20260811 09:15:49.986166 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.966861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:38029 (local address 127.0.0.1:46510)
0811 09:15:49.968297 (+ 1436us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:49.968339 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:49.968507 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:49.978562 (+ 10055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:49.978570 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:49.978597 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:49.979060 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:49.979076 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.980981 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.980987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:49.983482 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:49.983501 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.985234 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.985240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:49.985386 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.985412 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 09:15:49.985483 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 09:15:49.985678 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":121,"thread_start_us":732,"threads_started":1}
I20260811 09:15:49.986512 3401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.968400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:41261 (local address 127.0.0.1:40214)
0811 09:15:49.977624 (+ 9224us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:49.977639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:49.978308 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:49.979216 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:49.979220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:49.979247 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:49.979611 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:49.979621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.981097 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.981102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:49.983003 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:49.983016 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.985063 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.985067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:49.985251 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.985309 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 09:15:49.985452 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 09:15:49.985679 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9103,"mutex_wait_us":68,"thread_start_us":8298,"threads_started":1}
I20260811 09:15:49.986876 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.967060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46510 (local address 127.17.16.66:38029)
0811 09:15:49.977886 (+ 10826us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:49.977896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:49.977927 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:15:49.978029 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:49.978038 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:49.978297 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:49.979132 (+ 835us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:49.979325 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.980812 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.983651 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.985023 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.985159 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.985344 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:49.986425 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0811 09:15:49.986603 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10668,"thread_start_us":10568,"threads_started":1}
I20260811 09:15:49.987468 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.967409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:40149 (local address 127.0.0.1:58800)
0811 09:15:49.977006 (+ 9597us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:49.977025 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:49.977161 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:49.978802 (+ 1641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:49.978807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:49.978831 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:49.979327 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:49.979340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.981257 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.981262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:49.983506 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:49.983516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.986725 (+ 3209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.986731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:49.986887 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.986918 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:15:49.987013 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 09:15:49.987131 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9441,"mutex_wait_us":44,"thread_start_us":9955,"threads_started":1}
I20260811 09:15:49.987551 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:49.967726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.17.16.67:40149)
0811 09:15:49.978274 (+ 10548us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:49.978281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:49.978311 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:15:49.978401 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:49.978407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:49.978621 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:49.978812 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:49.979558 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.981115 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.985118 (+ 4003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:49.986449 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:49.986560 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:49.986770 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:49.987063 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 09:15:49.987228 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10394,"thread_start_us":10228,"threads_started":1}
I20260811 09:15:49.990592 3069 tablet_service.cc:1467] Processing CreateTablet for tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa (DEFAULT_TABLE table=test-table [id=4868349a82474e0e84779693af08bcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:49.990593 2862 tablet_service.cc:1467] Processing CreateTablet for tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa (DEFAULT_TABLE table=test-table [id=4868349a82474e0e84779693af08bcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:49.991453 3267 tablet_service.cc:1467] Processing CreateTablet for tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa (DEFAULT_TABLE table=test-table [id=4868349a82474e0e84779693af08bcfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:15:49.992158 3069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:49.992161 2862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:49.992586 3267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:15:50.003669 3411 tablet_bootstrap.cc:492] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: Bootstrap starting.
I20260811 09:15:50.006268 3414 tablet_bootstrap.cc:492] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: Bootstrap starting.
I20260811 09:15:50.006990 3412 tablet_bootstrap.cc:492] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: Bootstrap starting.
I20260811 09:15:50.007915 3411 tablet_bootstrap.cc:654] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:50.009219 3411 log.cc:824] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:50.010012 3414 tablet_bootstrap.cc:654] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:50.010254 3412 tablet_bootstrap.cc:654] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: Neither blocks nor log segments found. Creating new log.
I20260811 09:15:50.011106 3414 log.cc:824] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:50.011358 3412 log.cc:824] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: Log is configured to *not* fsync() on all Append() calls
I20260811 09:15:50.012538 3411 tablet_bootstrap.cc:492] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: No bootstrap required, opened a new log
I20260811 09:15:50.012801 3411 ts_tablet_manager.cc:1397] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 09:15:50.014333 3414 tablet_bootstrap.cc:492] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: No bootstrap required, opened a new log
I20260811 09:15:50.014411 3412 tablet_bootstrap.cc:492] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: No bootstrap required, opened a new log
I20260811 09:15:50.014564 3414 ts_tablet_manager.cc:1397] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 09:15:50.014706 3412 ts_tablet_manager.cc:1397] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260811 09:15:50.021517 3414 raft_consensus.cc:348] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 } } peers { permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 } } peers { permanent_uuid: "07af50e086ea4ffe9eaf20b846f7218e" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40149 } }
I20260811 09:15:50.021867 3414 raft_consensus.cc:374] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:50.021962 3414 raft_consensus.cc:729] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd19073155b4887a7001a7ab56baf97, State: Initialized, Role: FOLLOWER
I20260811 09:15:50.022524 3414 consensus_queue.cc:260] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 } } peers { permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 } } peers { permanent_uuid: "07af50e086ea4ffe9eaf20b846f7218e" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40149 } }
I20260811 09:15:50.022895 3411 raft_consensus.cc:348] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 } } peers { permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 } } peers { permanent_uuid: "07af50e086ea4ffe9eaf20b846f7218e" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40149 } }
I20260811 09:15:50.023365 3411 raft_consensus.cc:374] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:50.023490 3411 raft_consensus.cc:729] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07af50e086ea4ffe9eaf20b846f7218e, State: Initialized, Role: FOLLOWER
I20260811 09:15:50.023547 3159 heartbeater.cc:503] Master 127.17.16.126:37157 was elected leader, sending a full tablet report...
I20260811 09:15:50.024242 3414 ts_tablet_manager.cc:1428] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260811 09:15:50.024158 3411 consensus_queue.cc:260] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 } } peers { permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 } } peers { permanent_uuid: "07af50e086ea4ffe9eaf20b846f7218e" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40149 } }
I20260811 09:15:50.024816 3412 raft_consensus.cc:348] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 } } peers { permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 } } peers { permanent_uuid: "07af50e086ea4ffe9eaf20b846f7218e" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40149 } }
I20260811 09:15:50.025214 3374 heartbeater.cc:503] Master 127.17.16.126:37157 was elected leader, sending a full tablet report...
I20260811 09:15:50.025269 3412 raft_consensus.cc:374] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:15:50.025359 3412 raft_consensus.cc:729] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06cf4c45f7fe4519adc05ef37c29db92, State: Initialized, Role: FOLLOWER
I20260811 09:15:50.025857 3411 ts_tablet_manager.cc:1428] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 09:15:50.025910 3412 consensus_queue.cc:260] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 } } peers { permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 } } peers { permanent_uuid: "07af50e086ea4ffe9eaf20b846f7218e" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40149 } }
I20260811 09:15:50.027825 3412 ts_tablet_manager.cc:1428] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260811 09:15:50.080760 3423 raft_consensus.cc:482] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:15:50.081099 3423 raft_consensus.cc:504] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 } } peers { permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 } } peers { permanent_uuid: "07af50e086ea4ffe9eaf20b846f7218e" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40149 } }
I20260811 09:15:50.082793 3423 leader_election.cc:288] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cd19073155b4887a7001a7ab56baf97 (127.17.16.66:38029), 06cf4c45f7fe4519adc05ef37c29db92 (127.17.16.65:41261)
I20260811 09:15:50.089311 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:50.083055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:38029 (local address 127.17.16.67:34807)
0811 09:15:50.083253 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:50.083275 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:50.083403 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:50.084021 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:50.084026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:50.084059 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:50.084256 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:50.084268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.085554 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.085558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:50.087353 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:50.087369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.088579 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.088584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:50.088767 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:50.088793 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 09:15:50.088873 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 09:15:50.089025 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 09:15:50.089305 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:50.083155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:34807 (local address 127.17.16.66:38029)
0811 09:15:50.083358 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:50.083363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:50.083384 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:15:50.083609 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:50.083614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:50.083701 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:50.083808 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:50.084414 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.085391 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.087519 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.088430 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.088568 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:50.088652 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:50.088934 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 09:15:50.089062 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:15:50.089856 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:50.083733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:53371 (local address 127.17.16.65:41261)
0811 09:15:50.083931 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:50.083936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:50.084394 (+ 458us) server_negotiation.cc:407] Connection header received
0811 09:15:50.084535 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:50.084540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:50.084611 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:50.084714 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:15:50.085174 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.085969 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.088156 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.088971 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.089115 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:50.089195 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:50.089513 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 09:15:50.089641 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:15:50.090236 3095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0effa7ae114e3b8ecc9d5739d6f0fa" candidate_uuid: "07af50e086ea4ffe9eaf20b846f7218e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd19073155b4887a7001a7ab56baf97" is_pre_election: true
I20260811 09:15:50.090402 3426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:50.083464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:41261 (local address 127.17.16.67:53371)
0811 09:15:50.084290 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 09:15:50.084306 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:15:50.084418 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:15:50.084718 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:15:50.084721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:15:50.084738 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:15:50.085040 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:50.085050 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.086114 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.086120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:50.088005 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:15:50.088020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.089168 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.089174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:15:50.089330 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:50.089353 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:15:50.090049 (+ 696us) client_negotiation.cc:240] Negotiation successful
0811 09:15:50.090154 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":1,"thread_start_us":488,"threads_started":1}
I20260811 09:15:50.090751 3095 raft_consensus.cc:2454] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07af50e086ea4ffe9eaf20b846f7218e in term 0.
I20260811 09:15:50.091374 3197 leader_election.cc:302] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07af50e086ea4ffe9eaf20b846f7218e, 0cd19073155b4887a7001a7ab56baf97; no voters:
I20260811 09:15:50.091310 2893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0effa7ae114e3b8ecc9d5739d6f0fa" candidate_uuid: "07af50e086ea4ffe9eaf20b846f7218e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06cf4c45f7fe4519adc05ef37c29db92" is_pre_election: true
I20260811 09:15:50.091729 3423 raft_consensus.cc:2781] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:15:50.091790 2893 raft_consensus.cc:2454] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07af50e086ea4ffe9eaf20b846f7218e in term 0.
I20260811 09:15:50.091897 3423 raft_consensus.cc:482] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:15:50.092015 3423 raft_consensus.cc:3037] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:50.094384 3423 raft_consensus.cc:504] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 } } peers { permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 } } peers { permanent_uuid: "07af50e086ea4ffe9eaf20b846f7218e" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40149 } }
I20260811 09:15:50.094887 3423 leader_election.cc:288] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [CANDIDATE]: Term 1 election: Requested vote from peers 0cd19073155b4887a7001a7ab56baf97 (127.17.16.66:38029), 06cf4c45f7fe4519adc05ef37c29db92 (127.17.16.65:41261)
I20260811 09:15:50.095323 3095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0effa7ae114e3b8ecc9d5739d6f0fa" candidate_uuid: "07af50e086ea4ffe9eaf20b846f7218e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd19073155b4887a7001a7ab56baf97"
I20260811 09:15:50.095378 2893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0effa7ae114e3b8ecc9d5739d6f0fa" candidate_uuid: "07af50e086ea4ffe9eaf20b846f7218e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06cf4c45f7fe4519adc05ef37c29db92"
I20260811 09:15:50.095539 3095 raft_consensus.cc:3037] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:50.095553 2893 raft_consensus.cc:3037] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:15:50.098651 2893 raft_consensus.cc:2454] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07af50e086ea4ffe9eaf20b846f7218e in term 1.
I20260811 09:15:50.098651 3095 raft_consensus.cc:2454] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07af50e086ea4ffe9eaf20b846f7218e in term 1.
I20260811 09:15:50.099020 3197 leader_election.cc:302] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06cf4c45f7fe4519adc05ef37c29db92, 07af50e086ea4ffe9eaf20b846f7218e; no voters:
I20260811 09:15:50.099277 3423 raft_consensus.cc:2781] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:15:50.099677 3423 raft_consensus.cc:686] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 1 LEADER]: Becoming Leader. State: Replica: 07af50e086ea4ffe9eaf20b846f7218e, State: Running, Role: LEADER
I20260811 09:15:50.100160 3423 consensus_queue.cc:237] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 } } peers { permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 } } peers { permanent_uuid: "07af50e086ea4ffe9eaf20b846f7218e" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40149 } }
I20260811 09:15:50.104768 2652 catalog_manager.cc:5495] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e reported cstate change: term changed from 0 to 1, leader changed from <none> to 07af50e086ea4ffe9eaf20b846f7218e (127.17.16.67). New cstate: current_term: 1 leader_uuid: "07af50e086ea4ffe9eaf20b846f7218e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07af50e086ea4ffe9eaf20b846f7218e" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40149 } health_report { overall_health: HEALTHY } } }
I20260811 09:15:50.129027 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:50.124190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.17.16.67:40149)
0811 09:15:50.124352 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:50.124356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:50.124373 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:15:50.124461 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:50.124464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:50.124567 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:50.124697 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:15:50.125039 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.125850 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.127165 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.127559 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.127651 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:50.128056 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:15:50.128571 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:15:50.128687 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:50.128724 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 09:15:50.128791 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 09:15:50.129726 2962 tablet.cc:2369] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:15:50.137148 2893 raft_consensus.cc:1261] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92 [term 1 FOLLOWER]: Refusing update from remote peer 07af50e086ea4ffe9eaf20b846f7218e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:50.137166 3095 raft_consensus.cc:1261] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97 [term 1 FOLLOWER]: Refusing update from remote peer 07af50e086ea4ffe9eaf20b846f7218e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:15:50.138113 3429 consensus_queue.cc:1035] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd19073155b4887a7001a7ab56baf97" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:15:50.138423 3423 consensus_queue.cc:1035] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [LEADER]: Connected to new peer: Peer: permanent_uuid: "06cf4c45f7fe4519adc05ef37c29db92" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 09:15:50.141754 3160 tablet.cc:2369] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:15:50.150695 3434 mvcc.cc:205] Tried to move back new op lower bound from 7317257216551510016 to 7317257216414081024. Current Snapshot: MvccSnapshot[applied={T|T < 7317257216551510016}]
I20260811 09:15:50.155788 3435 mvcc.cc:205] Tried to move back new op lower bound from 7317257216551510016 to 7317257216414081024. Current Snapshot: MvccSnapshot[applied={T|T < 7317257216551510016}]
I20260811 09:15:50.164264 2653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:50.164507 2653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:15:50.168103 2653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32a0857efd9c4f909800ea704e449d86: Sending DeleteTablet for 3 replicas of tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa
I20260811 09:15:50.169260 3069 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:50 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:46510
I20260811 09:15:50.169366 3267 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:50 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:58800
I20260811 09:15:50.169418 2862 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:15:50 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:40214
I20260811 09:15:50.172479 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 2754
I20260811 09:15:50.182243 3445 tablet_replica.cc:330] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: stopping tablet replica
I20260811 09:15:50.182909 3452 tablet_replica.cc:330] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: stopping tablet replica
I20260811 09:15:50.183614 3452 raft_consensus.cc:2229] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:15:50.184191 3445 raft_consensus.cc:2229] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:50.184828 3445 raft_consensus.cc:2258] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:50.185570 3452 raft_consensus.cc:2258] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:50.188048 3445 ts_tablet_manager.cc:1905] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:50.188720 3448 tablet_replica.cc:330] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: stopping tablet replica
I20260811 09:15:50.188076 3452 ts_tablet_manager.cc:1905] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:50.192276 3448 raft_consensus.cc:2229] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:15:50.192951 3448 raft_consensus.cc:2258] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:15:50.195372 3448 ts_tablet_manager.cc:1905] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:15:50.195412 3445 ts_tablet_manager.cc:1918] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:50.195659 3445 log.cc:1196] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/5c0effa7ae114e3b8ecc9d5739d6f0fa
I20260811 09:15:50.196259 3445 ts_tablet_manager.cc:1939] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 0cd19073155b4887a7001a7ab56baf97: Deleting consensus metadata
I20260811 09:15:50.198555 2638 catalog_manager.cc:4841] TS 0cd19073155b4887a7001a7ab56baf97 (127.17.16.66:38029): tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa (table test-table [id=4868349a82474e0e84779693af08bcfb]) successfully deleted
Aug 11 09:15:50 dist-test-slave-ncp9 krb5kdc[2581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:15:50.202374 3452 ts_tablet_manager.cc:1918] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:50.202591 3452 log.cc:1196] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/5c0effa7ae114e3b8ecc9d5739d6f0fa
I20260811 09:15:50.203168 3452 ts_tablet_manager.cc:1939] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 07af50e086ea4ffe9eaf20b846f7218e: Deleting consensus metadata
I20260811 09:15:50.203332 3448 ts_tablet_manager.cc:1918] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:15:50.203852 3448 log.cc:1196] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/5c0effa7ae114e3b8ecc9d5739d6f0fa
I20260811 09:15:50.204461 3448 ts_tablet_manager.cc:1939] T 5c0effa7ae114e3b8ecc9d5739d6f0fa P 06cf4c45f7fe4519adc05ef37c29db92: Deleting consensus metadata
I20260811 09:15:50.205049 2639 catalog_manager.cc:4841] TS 07af50e086ea4ffe9eaf20b846f7218e (127.17.16.67:40149): tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa (table test-table [id=4868349a82474e0e84779693af08bcfb]) successfully deleted
I20260811 09:15:50.206303 2638 catalog_manager.cc:4841] TS 06cf4c45f7fe4519adc05ef37c29db92 (127.17.16.65:41261): tablet 5c0effa7ae114e3b8ecc9d5739d6f0fa (table test-table [id=4868349a82474e0e84779693af08bcfb]) successfully deleted
I20260811 09:15:50.209867 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:50.175495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.17.16.65:41261)
0811 09:15:50.175691 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:50.175697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:50.190889 (+ 15192us) server_negotiation.cc:407] Connection header received
0811 09:15:50.191386 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:50.191393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:50.191490 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:50.191569 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:50.193526 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.194626 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.196426 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.197104 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.197220 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:50.202688 (+ 5468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:50.202702 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:50.202720 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:50.202767 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:50.207048 (+ 4281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:50.207711 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:50.207720 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:50.207724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:50.207892 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:50.208223 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:50.208228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:50.208232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:50.208705 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:50.209056 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:50.209238 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 09:15:50.209461 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:15:50.210711 2952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:50.300333 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 2754
I20260811 09:15:50.331736 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 2974
Aug 11 09:15:50 dist-test-slave-ncp9 krb5kdc[2581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:15:50.346724 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:50.332558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46518 (local address 127.17.16.66:38029)
0811 09:15:50.332733 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:50.332739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:50.332759 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:15:50.332989 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:50.332996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:50.333085 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:50.333190 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:50.334447 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.335192 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.336382 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.336788 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.336934 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:50.340986 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:50.341000 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:50.341017 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:50.341052 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:50.344045 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:50.344697 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:50.344705 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:50.344708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:50.344846 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:50.345222 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:50.345227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:50.345231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:50.345609 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:50.345877 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:50.346207 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 09:15:50.346411 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:15:50.347419 3152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:50.433545 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 2974
I20260811 09:15:50.464641 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 3165
Aug 11 09:15:50 dist-test-slave-ncp9 krb5kdc[2581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:15:50.480569 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:15:50.465488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.17.16.67:40149)
0811 09:15:50.465705 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 09:15:50.465736 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 09:15:50.465757 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:15:50.465927 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:15:50.465931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:15:50.465997 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:15:50.466096 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:15:50.467459 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.468278 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.469555 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:15:50.470006 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:15:50.470121 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:15:50.474726 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:15:50.474739 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:15:50.474755 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:15:50.474788 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:15:50.477851 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:50.478620 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:50.478627 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:50.478629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:50.478771 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:15:50.479145 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:15:50.479148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:15:50.479150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:15:50.479506 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:15:50.479726 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 09:15:50.480066 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 09:15:50.480241 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 09:15:50.481263 3365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:50.602265 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 3165
I20260811 09:15:50.628715 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 2602
I20260811 09:15:50.630046 2702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:15:50.738399 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 2602
2026-08-11T09:15:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4440 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:15:50 dist-test-slave-ncp9 krb5kdc[3463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:15:50 dist-test-slave-ncp9 krb5kdc[3463](info): set up 2 sockets
Aug 11 09:15:50 dist-test-slave-ncp9 krb5kdc[3463](info): commencing operation
krb5kdc: starting...
W20260811 09:15:51.970518 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:15:52 dist-test-slave-ncp9 krb5kdc[3463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:15:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:15:52Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/rangeradmin_127.17.16.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/rangeradmin_127.17.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/rangerlookup_127.17.16.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/rangerlookup_127.17.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/HTTP_127.17.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/HTTP_127.17.16.84@KRBTEST.COM.keytab.
I20260811 09:15:52.329510 17473 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskNGP8Lb/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/postgres -l logfile start
2026-08-11 09:15:54.662 UTC [3513] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 09:15:54.662 UTC [3513] LOG: listening on IPv4 address "127.17.16.84", port 47113
2026-08-11 09:15:54.672 UTC [3513] LOG: listening on Unix socket "/tmp/.s.PGSQL.47113"
2026-08-11 09:15:54.693 UTC [3517] LOG: database system was shut down at 2026-08-11 09:15:53 UTC
2026-08-11 09:15:54.697 UTC [3513] LOG: database system is ready to accept connections
I20260811 09:15:55.789379 17473 mini_postgres.cc:96] Postgres bound to 47113
2026-08-11 09:15:55.814 UTC [3528] FATAL: database "slave" does not exist
127.17.16.84:47113 - accepting connections
I20260811 09:15:55.815109 17473 mini_ranger.cc:161] Starting Ranger...
I20260811 09:15:55.846143 17473 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 09:15:56.264196 17473 mini_ranger.cc:87] Created ranger Postgres database
I20260811 09:15:56.264357 17473 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-admin
2026-08-11 09:15:56,659 [I] DB FLAVOR :POSTGRES
2026-08-11 09:15:56,659 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:15:56,659 [I] Checking connection..
2026-08-11 09:15:56,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 09:15:56,948 [I] Checking connection passed.
2026-08-11 09:15:56,948 [I] --------- Verifying version history table ---------
2026-08-11 09:15:56,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:15:57,187 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:15:57,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:15:57,427 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:15:57,428 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 09:15:57,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 09:15:57.678 UTC [3630] WARNING: there is no transaction in progress
2026-08-11 09:15:57,686 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 09:15:57,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:15:57,947 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 09:15:57,948 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:15:57,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 09:15:58,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 09:15:58,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ncp9.c.gcp-upstream.internal','N') ;"
2026-08-11 09:15:58,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:15:58,923 [I] Table x_portal_user does not exist in database ranger
2026-08-11 09:15:58,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:15:59,181 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 09:15:59,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:15:59,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:15:59,680 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 09:15:59,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 09:16:00.588 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.604 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.619 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.634 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.649 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.667 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.731 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.738 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.751 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.760 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.769 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.783 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.792 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.798 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.804 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:00.811 UTC [3810] WARNING: there is no transaction in progress
2026-08-11 09:16:01,396 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 09:16:01,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 09:16:01,643 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 09:16:01,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:16:01,911 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 09:16:01,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:16:02,176 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 09:16:02,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:16:02,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:16:02,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ncp9.c.gcp-upstream.internal';"
2026-08-11 09:16:02,925 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 09:16:02,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 09:16:03,162 [I] DB_PATCHES have already been applied
I20260811 09:16:03.168198 17473 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-admin:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskNGP8Lb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 09:16:03.168310 17473 mini_ranger.cc:193] Using host: 127.17.16.84
I20260811 09:16:03.190099 17473 mini_ranger.cc:227] Ranger admin URL: http://127.17.16.84:46483
Aug 11, 2026 9:16:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:16:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:16:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 9:16:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 9:16:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 9:16:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 9:16:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 09:16:04.508378 17473 mini_ranger.cc:160] Time spent starting Ranger: real 8.693s user 0.000s sys 0.063s
Aug 11, 2026 9:16:24 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 9:16:24 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 9:16:24 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 9:16:24 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 9:16:25 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 09:16:25.621614 17473 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/rangerkms_127.17.16.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/rangerkms_127.17.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.16.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.16.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.16.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/HTTP_127.17.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.16.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/HTTP_127.17.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 09:16:25 dist-test-slave-ncp9 krb5kdc[3463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439785, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 09:16:25.848340 17473 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 09:16:25.875555 17473 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 09:16:26.634246 17473 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 09:16:26.634359 17473 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-kms
I20260811 09:16:26.634495 17473 mini_ranger_kms.cc:227] Using postgres at 127.17.16.84:47113
2026-08-11 09:16:27,286 [I] DB FLAVOR :POSTGRES
2026-08-11 09:16:27,286 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:16:27,286 [I] Checking connection
2026-08-11 09:16:27,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 09:16:27,529 [I] connection success
2026-08-11 09:16:27,529 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 09:16:27,529 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 09:16:27,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 09:16:27,756 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 09:16:27,756 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:16:27,756 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 09:16:27,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:47113/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 09:16:28,008 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 09:16:28.491281 17473 mini_ranger_kms.cc:310] Created kms service
I20260811 09:16:28.661903 17473 mini_ranger_kms.cc:326] Created kudu user
I20260811 09:16:28.705575 17473 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 09:16:28 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439785, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:16:29.108080 17473 mini_ranger_kms.cc:384] Added ranger policy
I20260811 09:16:29.109472 17473 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-kms:/tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 09:16:29.109537 17473 mini_ranger_kms.cc:240] Using host: 127.17.16.84
I20260811 09:16:29.142009 17473 mini_ranger_kms.cc:276] Ranger KMS PID: 4097
I20260811 09:16:29.142186 17473 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.16.84:33183
Aug 11, 2026 9:16:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:16:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:16:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 9:16:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 9:16:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 9:16:30 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 09:16:31 dist-test-slave-ncp9 krb5kdc[3463](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439791, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.16.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:16:31.781627 17473 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.933s user 0.129s sys 0.419s
I20260811 09:16:31.782027 17473 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 09:16:31.782308 17473 mini_ranger_kms.cc:401] 127.17.16.84:33183/kms/v1/keys
Aug 11 09:16:31 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439791, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.16.84@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
Aug 11 09:16:32 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439791, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.16.84@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
Aug 11 09:16:34 dist-test-slave-ncp9 krb5kdc[3463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:16:34.580570 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:42825
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:42825
--ranger_config_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:33183/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:16:35.007201 4186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:16:35.007539 4186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:16:35.007704 4186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:16:35.016830 4186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:16:35.016955 4186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:16:35.017016 4186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:16:35.017074 4186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:16:35.017154 4186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:16:35.029664 4186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.16.84:33183/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:42825
--ranger_config_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:42825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:16:35.030966 4186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:16:35.032297 4186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:16:35.048719 4201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:16:35.048784 4207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:16:35.048756 4202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:16:35.049556 4186 server_base.cc:1034] running on GCE node
I20260811 09:16:35.050527 4186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:16:35.052435 4186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:16:35.053649 4186 hybrid_clock.cc:648] HybridClock initialized: now 1786439795053604 us; error 33 us; skew 500 ppm
I20260811 09:16:35.053995 4186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:16:35 dist-test-slave-ncp9 krb5kdc[3463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439795, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:16:35.061357 4186 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:16:35.063437 4186 webserver.cc:460] Webserver started at http://127.17.16.126:36821/ using document root <none> and password file <none>
I20260811 09:16:35.064072 4186 fs_manager.cc:359] Metadata directory not provided
I20260811 09:16:35.064186 4186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:16:35.064486 4186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:16:35 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439795, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:16:35.116371 4186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "91398aefd574415fb211c5d077d43d8f"
format_stamp: "Formatted at 2026-08-11 09:16:35 on dist-test-slave-ncp9"
server_key: "d50a90864c3b73d3e10621f6d16e5d94"
server_key_iv: "ee0538a56468e3728d8073ff86867755"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:35.117372 4186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "91398aefd574415fb211c5d077d43d8f"
format_stamp: "Formatted at 2026-08-11 09:16:35 on dist-test-slave-ncp9"
server_key: "d50a90864c3b73d3e10621f6d16e5d94"
server_key_iv: "ee0538a56468e3728d8073ff86867755"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:35.125349 4186 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260811 09:16:35.146899 4215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:16:35.148351 4186 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:16:35.148510 4186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "91398aefd574415fb211c5d077d43d8f"
format_stamp: "Formatted at 2026-08-11 09:16:35 on dist-test-slave-ncp9"
server_key: "d50a90864c3b73d3e10621f6d16e5d94"
server_key_iv: "ee0538a56468e3728d8073ff86867755"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:35.148729 4186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:16:35.168458 4186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:16:35.173285 4186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:16:35.173913 4186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:16:35.206696 4186 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:42825
I20260811 09:16:35.206722 4296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:42825 every 8 connection(s)
I20260811 09:16:35.208062 4186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:16:35.212671 4301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:16:35.216536 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 4186
I20260811 09:16:35.216753 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:16:35.217790 17473 external_mini_cluster.cc:1428] Setting key ff20baac661159f9cb2c0bdcfb4477be
I20260811 09:16:35.228127 4301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f: Bootstrap starting.
I20260811 09:16:35.232465 4301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f: Neither blocks nor log segments found. Creating new log.
I20260811 09:16:35.233733 4301 log.cc:824] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:16:35 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:16:35.237458 4301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f: No bootstrap required, opened a new log
I20260811 09:16:35.243597 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:35.220031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.17.16.126:42825)
0811 09:16:35.220787 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:35.220799 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:35.220838 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:16:35.223401 (+ 2563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:35.223424 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:35.223798 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:35.224109 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:16:35.226227 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:35.227756 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:35.229469 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:35.230381 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:35.230535 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:35.234937 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:16:35.234945 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:16:35.234967 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:16:35.235009 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:16:35.238437 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:35.239153 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:35.239164 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:35.239173 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:35.239399 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:35.239832 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:35.239838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:35.239841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:35.240587 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:16:35.240928 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:35.241832 (+ 904us) server_negotiation.cc:299] Negotiation successful
0811 09:16:35.242266 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":361,"threads_started":1}
I20260811 09:16:35.249647 4301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91398aefd574415fb211c5d077d43d8f" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42825 } }
I20260811 09:16:35.249994 4301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:16:35.250067 4301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91398aefd574415fb211c5d077d43d8f, State: Initialized, Role: FOLLOWER
I20260811 09:16:35.250594 4301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91398aefd574415fb211c5d077d43d8f" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42825 } }
I20260811 09:16:35.250800 4301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:16:35.250900 4301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:16:35.251015 4301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:16:35.253726 4301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91398aefd574415fb211c5d077d43d8f" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42825 } }
I20260811 09:16:35.254195 4301 leader_election.cc:302] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91398aefd574415fb211c5d077d43d8f; no voters:
I20260811 09:16:35.255177 4301 leader_election.cc:288] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:16:35.255221 4307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:16:35.256752 4307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [term 1 LEADER]: Becoming Leader. State: Replica: 91398aefd574415fb211c5d077d43d8f, State: Running, Role: LEADER
I20260811 09:16:35.257735 4301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:16:35.257736 4307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91398aefd574415fb211c5d077d43d8f" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42825 } }
I20260811 09:16:35.269071 4312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91398aefd574415fb211c5d077d43d8f. Latest consensus state: current_term: 1 leader_uuid: "91398aefd574415fb211c5d077d43d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91398aefd574415fb211c5d077d43d8f" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42825 } } }
I20260811 09:16:35.268987 4310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91398aefd574415fb211c5d077d43d8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91398aefd574415fb211c5d077d43d8f" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 42825 } } }
I20260811 09:16:35.269424 4310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [sys.catalog]: This master's current role is: LEADER
I20260811 09:16:35.269423 4312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f [sys.catalog]: This master's current role is: LEADER
I20260811 09:16:35.270417 4301 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 09:16:35.271075 4339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:16:35.287158 4339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:16:35.308851 4339 catalog_manager.cc:1347] Generated new cluster ID: 042dd7f6da034a9d887bb0d776632e8f
I20260811 09:16:35.309229 4339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:16:35.329243 4339 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:16:35.332021 4339 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:16:35.339025 4339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f: Generated new TSK 0
I20260811 09:16:35.339830 4339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:16:36.196609 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:33183/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:42825
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:16:36 dist-test-slave-ncp9 krb5kdc[3463](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 09:16:36.847149 4386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:16:36.847720 4386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:16:36.847993 4386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:16:36.862128 4386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:16:36.862437 4386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:16:36.862793 4386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:16:36.883105 4386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.16.84:33183/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:42825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:16:36.885217 4386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:16:36.887727 4386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:16:36.914587 4400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:16:36.918449 4401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:16:36.936322 4407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:16:36.938339 4386 server_base.cc:1034] running on GCE node
I20260811 09:16:36.939419 4386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:16:36.940834 4386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:16:36.942185 4386 hybrid_clock.cc:648] HybridClock initialized: now 1786439796942110 us; error 59 us; skew 500 ppm
I20260811 09:16:36.942616 4386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:16:36 dist-test-slave-ncp9 krb5kdc[3463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:16:36.948434 4386 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:16:36.952284 4386 webserver.cc:460] Webserver started at http://127.17.16.65:40965/ using document root <none> and password file <none>
I20260811 09:16:36.953423 4386 fs_manager.cc:359] Metadata directory not provided
I20260811 09:16:36.953665 4386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:16:36.954206 4386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:16:36 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:16:36.978401 4386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "03183dc24eb84c1ea1853e788ac2f191"
format_stamp: "Formatted at 2026-08-11 09:16:36 on dist-test-slave-ncp9"
server_key: "f9aa25ce66f7249427baebf443062015"
server_key_iv: "418d087a4c9ecf8dd181a1d3d5754235"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:36.980346 4386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "03183dc24eb84c1ea1853e788ac2f191"
format_stamp: "Formatted at 2026-08-11 09:16:36 on dist-test-slave-ncp9"
server_key: "f9aa25ce66f7249427baebf443062015"
server_key_iv: "418d087a4c9ecf8dd181a1d3d5754235"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:36.991446 4386 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.003s sys 0.006s
Aug 11 09:16:36 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:16:37.030046 4418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:16:37.034171 4386 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.005s sys 0.006s
I20260811 09:16:37.034590 4386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "03183dc24eb84c1ea1853e788ac2f191"
format_stamp: "Formatted at 2026-08-11 09:16:36 on dist-test-slave-ncp9"
server_key: "f9aa25ce66f7249427baebf443062015"
server_key_iv: "418d087a4c9ecf8dd181a1d3d5754235"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:37.035112 4386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:16:37.118460 4386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:16:37.128913 4386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:16:37.129626 4386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:16:37.143285 4386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:16:37.163226 4386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:16:37.163486 4386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 09:16:37.181937 4386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:16:37.182271 4386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
Aug 11 09:16:37 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:16:37.666100 4594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:45229 every 8 connection(s)
I20260811 09:16:37.666545 4386 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:45229
I20260811 09:16:37.668540 4386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:16:37.674249 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 4386
I20260811 09:16:37.674490 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:16:37.675796 17473 external_mini_cluster.cc:1428] Setting key d3800fe44cdd0ebe0d90c1de692c0a3f
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Aug 11 09:16:37 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439796, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:16:37.758836 4598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:37.684772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:42825 (local address 127.17.16.65:33623)
0811 09:16:37.702438 (+ 17666us) negotiation.cc:107] Waiting for socket to connect
0811 09:16:37.702509 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 09:16:37.708198 (+ 5689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:16:37.721319 (+ 13121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:16:37.721334 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:16:37.722076 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:16:37.723419 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:37.723452 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:37.734846 (+ 11394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:37.734853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:37.737615 (+ 2762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:37.737635 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:37.739186 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:37.739198 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:37.739654 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:37.740645 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:16:37.740676 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:16:37.744178 (+ 3502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:16:37.753721 (+ 9543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:16:37.753731 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:16:37.753751 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:16:37.754516 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:16:37.755131 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:16:37.755136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:16:37.755140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:16:37.755393 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:16:37.756089 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:16:37.756099 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:16:37.756554 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 09:16:37.756850 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 09:16:37.757432 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17301,"thread_start_us":19966,"threads_started":1}
I20260811 09:16:37.760067 4597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:37.683841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:33623 (local address 127.17.16.126:42825)
0811 09:16:37.711481 (+ 27640us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:37.711488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:37.711511 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:16:37.711582 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:37.711588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:37.711688 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:37.711816 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:16:37.733080 (+ 21264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:37.734599 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:37.737790 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:37.738801 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:37.738965 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:37.749195 (+ 10230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:16:37.749207 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:16:37.749213 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:16:37.749265 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:16:37.753547 (+ 4282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:37.754701 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:37.754706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:37.754710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:37.754969 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:37.755554 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:37.755559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:37.755563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:37.755929 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:16:37.759305 (+ 3376us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:37.759439 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 09:16:37.759714 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27492,"thread_start_us":27271,"threads_started":1}
I20260811 09:16:37.762071 4595 heartbeater.cc:347] Connected to a master server at 127.17.16.126:42825
I20260811 09:16:37.762444 4595 heartbeater.cc:464] Registering TS with master...
I20260811 09:16:37.763556 4595 heartbeater.cc:511] Master 127.17.16.126:42825 requested a full tablet report, sending...
I20260811 09:16:37.766885 4240 ts_manager.cc:194] Registered new tserver with Master: 03183dc24eb84c1ea1853e788ac2f191 (127.17.16.65:45229)
I20260811 09:16:37.769071 4240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:33623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
W20260811 09:16:37.780570 17473 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.17.16.66: real 0.101s user 0.001s sys 0.024s
I20260811 09:16:37.781490 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:33183/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:42825
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:16:38.386734 4606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:16:38.387036 4606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:16:38.387179 4606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:16:38.398253 4606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:16:38.398597 4606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:16:38.398872 4606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:16:38.412757 4606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.16.84:33183/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:42825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:16:38.414341 4606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:16:38.415880 4606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:16:38.437189 4626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:16:38.440472 4627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:16:38.441675 4631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:16:38.443115 4606 server_base.cc:1034] running on GCE node
I20260811 09:16:38.443905 4606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:16:38.445219 4606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:16:38.446463 4606 hybrid_clock.cc:648] HybridClock initialized: now 1786439798446383 us; error 76 us; skew 500 ppm
I20260811 09:16:38.446779 4606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:16:38 dist-test-slave-ncp9 krb5kdc[3463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439798, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:16:38.451535 4606 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:16:38.453816 4606 webserver.cc:460] Webserver started at http://127.17.16.66:44851/ using document root <none> and password file <none>
I20260811 09:16:38.454869 4606 fs_manager.cc:359] Metadata directory not provided
I20260811 09:16:38.455088 4606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:16:38.455549 4606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:16:38 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439798, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:16:38.476233 4606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "97bfe6e3e29e4fdcbda8f28bbe238979"
format_stamp: "Formatted at 2026-08-11 09:16:38 on dist-test-slave-ncp9"
server_key: "9ae249aeeef7a202de4491170477b159"
server_key_iv: "564fba45a59be2800b19fe0dbb916ad6"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:38.477435 4606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "97bfe6e3e29e4fdcbda8f28bbe238979"
format_stamp: "Formatted at 2026-08-11 09:16:38 on dist-test-slave-ncp9"
server_key: "9ae249aeeef7a202de4491170477b159"
server_key_iv: "564fba45a59be2800b19fe0dbb916ad6"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:38.487185 4606 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.008s
I20260811 09:16:38.508565 4638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:16:38.514604 4606 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.001s
I20260811 09:16:38.514876 4606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "97bfe6e3e29e4fdcbda8f28bbe238979"
format_stamp: "Formatted at 2026-08-11 09:16:38 on dist-test-slave-ncp9"
server_key: "9ae249aeeef7a202de4491170477b159"
server_key_iv: "564fba45a59be2800b19fe0dbb916ad6"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:38.515232 4606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:16:38.561364 4606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:16:38.567675 4606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:16:38.568198 4606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:16:38.569998 4606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:16:38.573202 4606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:16:38.573469 4606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:16:38.573745 4606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:16:38.573901 4606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:16:38.737326 4606 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:44413
I20260811 09:16:38.737403 4800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:44413 every 8 connection(s)
I20260811 09:16:38.739161 4606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:16:38.739459 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 4606
I20260811 09:16:38.739638 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:16:38.740182 17473 external_mini_cluster.cc:1428] Setting key b0c86384c4dd8828f46ebb3d2e5d9b73
I20260811 09:16:38.772740 4595 heartbeater.cc:503] Master 127.17.16.126:42825 was elected leader, sending a full tablet report...
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Aug 11 09:16:38 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439798, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:16:38.797780 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:38.745146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:51335 (local address 127.17.16.126:42825)
0811 09:16:38.767083 (+ 21937us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:38.767090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:38.767232 (+ 142us) server_negotiation.cc:407] Connection header received
0811 09:16:38.771791 (+ 4559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:38.771799 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:38.771909 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:38.772065 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:16:38.777947 (+ 5882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:38.779268 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:38.781835 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:38.782684 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:38.782841 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:38.788321 (+ 5480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:16:38.788329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:16:38.788334 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:16:38.788411 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:16:38.793224 (+ 4813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:38.794291 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:38.794296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:38.794300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:38.794555 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:38.795250 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:38.795255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:38.795258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:38.795738 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:16:38.795961 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:38.797057 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0811 09:16:38.797320 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21777,"thread_start_us":21647,"threads_started":1}
I20260811 09:16:38.799170 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:38.744558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:42825 (local address 127.17.16.66:51335)
0811 09:16:38.767001 (+ 22443us) negotiation.cc:107] Waiting for socket to connect
0811 09:16:38.767087 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 09:16:38.771298 (+ 4211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:16:38.775408 (+ 4110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:16:38.775423 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:16:38.776273 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:16:38.777716 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:38.777749 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:38.779463 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:38.779468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:38.781653 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:38.781670 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:38.783436 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:38.783444 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:38.783724 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:38.784560 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:16:38.784591 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:16:38.787968 (+ 3377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:16:38.793386 (+ 5418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:16:38.793395 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:16:38.793413 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:16:38.794148 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:16:38.794687 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:16:38.794691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:16:38.794695 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:16:38.795078 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:16:38.796048 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:16:38.796057 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:16:38.796538 (+ 481us) client_negotiation.cc:769] Sending connection context
0811 09:16:38.796840 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 09:16:38.797528 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21392,"thread_start_us":20896,"threads_started":1}
I20260811 09:16:38.801878 4801 heartbeater.cc:347] Connected to a master server at 127.17.16.126:42825
I20260811 09:16:38.802237 4801 heartbeater.cc:464] Registering TS with master...
I20260811 09:16:38.802990 4801 heartbeater.cc:511] Master 127.17.16.126:42825 requested a full tablet report, sending...
I20260811 09:16:38.804651 4240 ts_manager.cc:194] Registered new tserver with Master: 97bfe6e3e29e4fdcbda8f28bbe238979 (127.17.16.66:44413)
I20260811 09:16:38.805836 4240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:51335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:16:38.822469 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:33183/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:42825
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:16:39.493026 4814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:16:39.493462 4814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:16:39.493636 4814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:16:39.504263 4814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:16:39.504463 4814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:16:39.504657 4814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:16:39.519203 4814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.16.84:33183/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:42825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:16:39.520609 4814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:16:39.522323 4814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:16:39.540999 4822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:16:39.541124 4821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:16:39.541257 4824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:16:39.542244 4814 server_base.cc:1034] running on GCE node
I20260811 09:16:39.542920 4814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:16:39.543923 4814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:16:39.545186 4814 hybrid_clock.cc:648] HybridClock initialized: now 1786439799545063 us; error 44 us; skew 500 ppm
I20260811 09:16:39.545630 4814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:16:39 dist-test-slave-ncp9 krb5kdc[3463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439799, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:16:39.550549 4814 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:16:39.552322 4814 webserver.cc:460] Webserver started at http://127.17.16.67:37217/ using document root <none> and password file <none>
I20260811 09:16:39.553258 4814 fs_manager.cc:359] Metadata directory not provided
I20260811 09:16:39.553519 4814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:16:39.553963 4814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:16:39 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439799, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:16:39.572106 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "8952dc51d0204397bdc74cea948a5799"
format_stamp: "Formatted at 2026-08-11 09:16:39 on dist-test-slave-ncp9"
server_key: "bc6ba5886d243de3d8575398b671ad5f"
server_key_iv: "16cc0dddd1bbbc518b9184710953d0aa"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:39.573201 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "8952dc51d0204397bdc74cea948a5799"
format_stamp: "Formatted at 2026-08-11 09:16:39 on dist-test-slave-ncp9"
server_key: "bc6ba5886d243de3d8575398b671ad5f"
server_key_iv: "16cc0dddd1bbbc518b9184710953d0aa"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:39.580267 4814 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260811 09:16:39.597329 4832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:16:39.599257 4814 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:16:39.599583 4814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "8952dc51d0204397bdc74cea948a5799"
format_stamp: "Formatted at 2026-08-11 09:16:39 on dist-test-slave-ncp9"
server_key: "bc6ba5886d243de3d8575398b671ad5f"
server_key_iv: "16cc0dddd1bbbc518b9184710953d0aa"
server_key_version: "kuduclusterkey@0"
I20260811 09:16:39.599900 4814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:16:39.619068 4814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:16:39.624750 4814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:16:39.625269 4814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:16:39.626893 4814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:16:39.629328 4814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:16:39.629444 4814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:16:39.629679 4814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:16:39.629824 4814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:16:39.721182 4814 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:38899
I20260811 09:16:39.721210 4999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:38899 every 8 connection(s)
I20260811 09:16:39.722709 4814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:16:39.728439 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 4814
I20260811 09:16:39.728624 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:16:39.729219 17473 external_mini_cluster.cc:1428] Setting key 96418fa2470e17c9f27d79b29c5b8775
Aug 11 09:16:39 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439799, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:16:39.749778 5002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:39.726499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:59049 (local address 127.17.16.126:42825)
0811 09:16:39.727132 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:39.727137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:39.728670 (+ 1533us) server_negotiation.cc:407] Connection header received
0811 09:16:39.731375 (+ 2705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:39.731379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:39.731510 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:39.731628 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:16:39.733851 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.734627 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.736574 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.737584 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.737722 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:39.741887 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:16:39.741892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:16:39.741896 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:16:39.741929 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:16:39.746294 (+ 4365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:39.747183 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:39.747188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:39.747191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:39.747372 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:39.747865 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:39.747870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:39.747872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:39.748347 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:16:39.748533 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:39.749140 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 09:16:39.749303 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":320,"threads_started":1}
I20260811 09:16:39.750541 5005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:39.727198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:42825 (local address 127.17.16.67:59049)
0811 09:16:39.728475 (+ 1277us) negotiation.cc:107] Waiting for socket to connect
0811 09:16:39.728553 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 09:16:39.731156 (+ 2603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:16:39.731898 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:16:39.731914 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:16:39.732809 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:16:39.733686 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:39.733706 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.734748 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.734752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:39.736383 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:39.736395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.737821 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.737827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:39.738053 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:39.738832 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:16:39.738854 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:16:39.741613 (+ 2759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:16:39.746415 (+ 4802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:16:39.746421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:16:39.746432 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:16:39.747062 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:16:39.747452 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:16:39.747455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:16:39.747457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:16:39.747724 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:16:39.748447 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:16:39.748453 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:16:39.748785 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 09:16:39.749041 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 09:16:39.749430 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"thread_start_us":790,"threads_started":1}
I20260811 09:16:39.751909 5000 heartbeater.cc:347] Connected to a master server at 127.17.16.126:42825
I20260811 09:16:39.752225 5000 heartbeater.cc:464] Registering TS with master...
I20260811 09:16:39.753337 5000 heartbeater.cc:511] Master 127.17.16.126:42825 requested a full tablet report, sending...
I20260811 09:16:39.754861 4240 ts_manager.cc:194] Registered new tserver with Master: 8952dc51d0204397bdc74cea948a5799 (127.17.16.67:38899)
I20260811 09:16:39.755986 4240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:59049
I20260811 09:16:39.757874 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:16:39.780393 5002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:39.763976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53688 (local address 127.17.16.126:42825)
0811 09:16:39.764164 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:39.764171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:39.765146 (+ 975us) server_negotiation.cc:407] Connection header received
0811 09:16:39.765303 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:39.765307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:39.765377 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:39.765452 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:16:39.767611 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.768510 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.770239 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.770903 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.770996 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:39.773788 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:16:39.773793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:16:39.773796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:16:39.773827 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:16:39.777406 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:39.778167 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:39.778172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:39.778174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:39.778309 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:39.778809 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:39.778813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:39.778815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:39.779222 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:16:39.779546 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:39.779801 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 09:16:39.780027 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:16:39.785120 4240 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:16:39.797216 4240 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:16:39.809614 4801 heartbeater.cc:503] Master 127.17.16.126:42825 was elected leader, sending a full tablet report...
I20260811 09:16:39.843585 5026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:39.812946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:38899 (local address 127.0.0.1:41996)
0811 09:16:39.825235 (+ 12289us) negotiation.cc:107] Waiting for socket to connect
0811 09:16:39.825290 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:16:39.825509 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:16:39.834020 (+ 8511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:16:39.834029 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:16:39.834061 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:16:39.835268 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:39.835284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.837603 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.837608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:39.840259 (+ 2651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:39.840273 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.842541 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.842547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:39.842729 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:39.842783 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 09:16:39.842914 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 09:16:39.843132 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11987,"mutex_wait_us":28,"thread_start_us":15861,"threads_started":1}
I20260811 09:16:39.844285 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:39.813348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41996 (local address 127.17.16.67:38899)
0811 09:16:39.832761 (+ 19413us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:39.832768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:39.832825 (+ 57us) server_negotiation.cc:407] Connection header received
0811 09:16:39.832967 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:39.832977 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:39.833262 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:39.833520 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:16:39.835535 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.837456 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.840467 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.842012 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.842149 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:39.842402 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:39.843844 (+ 1442us) server_negotiation.cc:299] Negotiation successful
0811 09:16:39.843986 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19236,"thread_start_us":19059,"threads_started":1}
I20260811 09:16:39.845083 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:39.813054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45229 (local address 127.0.0.1:45152)
0811 09:16:39.829143 (+ 16089us) negotiation.cc:107] Waiting for socket to connect
0811 09:16:39.829163 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:16:39.829284 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:16:39.835537 (+ 6253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:16:39.835541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:16:39.835566 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:16:39.836491 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:39.836506 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.838814 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.838819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:39.841615 (+ 2796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:39.841629 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.844066 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.844073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:39.844281 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:39.844313 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:16:39.844383 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 09:16:39.844509 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15947,"thread_start_us":12609,"threads_started":1}
I20260811 09:16:39.845211 5032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:39.815497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60286 (local address 127.17.16.66:44413)
0811 09:16:39.833142 (+ 17645us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:39.833152 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:39.833909 (+ 757us) server_negotiation.cc:407] Connection header received
0811 09:16:39.834013 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:39.834032 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:39.834253 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:39.834433 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:16:39.835585 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.837134 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.841121 (+ 3987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.843130 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.843300 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:39.843653 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:39.844216 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 09:16:39.844598 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17465,"thread_start_us":17230,"threads_started":1}
I20260811 09:16:39.848698 4903 tablet_service.cc:1467] Processing CreateTablet for tablet 50f9a1c76e164089bae502ac698146a7 (DEFAULT_TABLE table=test-table [id=ad45aa37c9464341833daca863d7d1e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:16:39.845127 5031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:39.815406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:44413 (local address 127.0.0.1:60286)
0811 09:16:39.833624 (+ 18218us) negotiation.cc:107] Waiting for socket to connect
0811 09:16:39.833642 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:16:39.833782 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:16:39.834673 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:16:39.834677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:16:39.834701 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:16:39.835397 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:39.835413 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.837256 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.837261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:39.840873 (+ 3612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:39.840912 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.843501 (+ 2589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.843509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:39.844025 (+ 516us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:39.844061 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:16:39.844154 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 09:16:39.844282 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18063,"mutex_wait_us":21,"thread_start_us":17903,"threads_started":1}
I20260811 09:16:39.849716 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:39.813339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45152 (local address 127.17.16.65:45229)
0811 09:16:39.834902 (+ 21563us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:39.834911 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:39.834941 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:16:39.835077 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:39.835084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:39.835344 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:39.835568 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:16:39.836791 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.838643 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.841804 (+ 3161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:39.843215 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:39.843337 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:39.843659 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:39.845328 (+ 1669us) server_negotiation.cc:299] Negotiation successful
0811 09:16:39.849162 (+ 3834us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21358,"thread_start_us":20821,"threads_started":1}
I20260811 09:16:39.851095 4903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f9a1c76e164089bae502ac698146a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:16:39.853705 4489 tablet_service.cc:1467] Processing CreateTablet for tablet 50f9a1c76e164089bae502ac698146a7 (DEFAULT_TABLE table=test-table [id=ad45aa37c9464341833daca863d7d1e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:16:39.853803 4703 tablet_service.cc:1467] Processing CreateTablet for tablet 50f9a1c76e164089bae502ac698146a7 (DEFAULT_TABLE table=test-table [id=ad45aa37c9464341833daca863d7d1e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:16:39.855299 4489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f9a1c76e164089bae502ac698146a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:16:39.855412 4703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50f9a1c76e164089bae502ac698146a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:16:39.868983 5034 tablet_bootstrap.cc:492] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: Bootstrap starting.
I20260811 09:16:39.874174 5034 tablet_bootstrap.cc:654] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: Neither blocks nor log segments found. Creating new log.
I20260811 09:16:39.875937 5037 tablet_bootstrap.cc:492] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: Bootstrap starting.
I20260811 09:16:39.877341 5036 tablet_bootstrap.cc:492] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: Bootstrap starting.
I20260811 09:16:39.877768 5034 log.cc:824] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: Log is configured to *not* fsync() on all Append() calls
I20260811 09:16:39.881909 5037 tablet_bootstrap.cc:654] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: Neither blocks nor log segments found. Creating new log.
I20260811 09:16:39.882619 5036 tablet_bootstrap.cc:654] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: Neither blocks nor log segments found. Creating new log.
I20260811 09:16:39.882866 5034 tablet_bootstrap.cc:492] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: No bootstrap required, opened a new log
I20260811 09:16:39.883313 5034 ts_tablet_manager.cc:1397] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.005s
I20260811 09:16:39.884104 5037 log.cc:824] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: Log is configured to *not* fsync() on all Append() calls
I20260811 09:16:39.884341 5036 log.cc:824] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: Log is configured to *not* fsync() on all Append() calls
I20260811 09:16:39.887902 5037 tablet_bootstrap.cc:492] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: No bootstrap required, opened a new log
I20260811 09:16:39.888259 5037 ts_tablet_manager.cc:1397] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260811 09:16:39.888444 5036 tablet_bootstrap.cc:492] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: No bootstrap required, opened a new log
I20260811 09:16:39.888839 5036 ts_tablet_manager.cc:1397] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.009s
I20260811 09:16:39.896008 5034 raft_consensus.cc:348] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 } } peers { permanent_uuid: "03183dc24eb84c1ea1853e788ac2f191" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45229 } } peers { permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 } }
I20260811 09:16:39.896569 5034 raft_consensus.cc:374] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:16:39.896790 5034 raft_consensus.cc:729] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8952dc51d0204397bdc74cea948a5799, State: Initialized, Role: FOLLOWER
I20260811 09:16:39.897490 5034 consensus_queue.cc:260] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 } } peers { permanent_uuid: "03183dc24eb84c1ea1853e788ac2f191" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45229 } } peers { permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 } }
I20260811 09:16:39.899376 5000 heartbeater.cc:503] Master 127.17.16.126:42825 was elected leader, sending a full tablet report...
I20260811 09:16:39.900120 5034 ts_tablet_manager.cc:1428] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: Time spent starting tablet: real 0.017s user 0.019s sys 0.000s
I20260811 09:16:39.903505 5037 raft_consensus.cc:348] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 } } peers { permanent_uuid: "03183dc24eb84c1ea1853e788ac2f191" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45229 } } peers { permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 } }
I20260811 09:16:39.904083 5037 raft_consensus.cc:374] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:16:39.904253 5037 raft_consensus.cc:729] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97bfe6e3e29e4fdcbda8f28bbe238979, State: Initialized, Role: FOLLOWER
I20260811 09:16:39.904501 5036 raft_consensus.cc:348] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 } } peers { permanent_uuid: "03183dc24eb84c1ea1853e788ac2f191" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45229 } } peers { permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 } }
I20260811 09:16:39.905182 5036 raft_consensus.cc:374] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:16:39.905022 5037 consensus_queue.cc:260] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 } } peers { permanent_uuid: "03183dc24eb84c1ea1853e788ac2f191" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45229 } } peers { permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 } }
I20260811 09:16:39.905443 5036 raft_consensus.cc:729] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03183dc24eb84c1ea1853e788ac2f191, State: Initialized, Role: FOLLOWER
I20260811 09:16:39.906258 5036 consensus_queue.cc:260] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 } } peers { permanent_uuid: "03183dc24eb84c1ea1853e788ac2f191" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45229 } } peers { permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 } }
I20260811 09:16:39.908622 5037 ts_tablet_manager.cc:1428] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: Time spent starting tablet: real 0.020s user 0.016s sys 0.004s
I20260811 09:16:39.908627 5036 ts_tablet_manager.cc:1428] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: Time spent starting tablet: real 0.020s user 0.012s sys 0.006s
W20260811 09:16:39.959154 4600 tablet.cc:2369] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:16:39.975621 5001 tablet.cc:2369] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:16:40.009588 4803 tablet.cc:2369] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:16:40.040871 5046 raft_consensus.cc:482] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:16:40.041339 5046 raft_consensus.cc:504] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 } } peers { permanent_uuid: "03183dc24eb84c1ea1853e788ac2f191" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45229 } } peers { permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 } }
I20260811 09:16:40.043782 5046 leader_election.cc:288] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8952dc51d0204397bdc74cea948a5799 (127.17.16.67:38899), 97bfe6e3e29e4fdcbda8f28bbe238979 (127.17.16.66:44413)
I20260811 09:16:40.054050 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:40.044082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:54513 (local address 127.17.16.67:38899)
0811 09:16:40.044321 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:40.044328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:40.046613 (+ 2285us) server_negotiation.cc:407] Connection header received
0811 09:16:40.046887 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:40.046891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:40.047015 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:40.047152 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:16:40.048193 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.049142 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.051646 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.053009 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.053194 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:40.053280 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:40.053611 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 09:16:40.053735 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 09:16:40.054386 5047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:40.044760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:38899 (local address 127.17.16.65:54513)
0811 09:16:40.046491 (+ 1731us) negotiation.cc:107] Waiting for socket to connect
0811 09:16:40.046512 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:16:40.046726 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:16:40.047190 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:16:40.047195 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:16:40.047223 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:16:40.048032 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:40.048052 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.049275 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.049280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:40.051496 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:40.051510 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.053334 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.053341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:40.053457 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:40.053481 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:16:40.053565 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 09:16:40.053760 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1541,"thread_start_us":1199,"threads_started":1}
I20260811 09:16:40.054780 5032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:40.044193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:47233 (local address 127.17.16.66:44413)
0811 09:16:40.044457 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:40.044464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:40.046862 (+ 2398us) server_negotiation.cc:407] Connection header received
0811 09:16:40.047175 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:40.047179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:40.047241 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:40.047320 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:16:40.048551 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.049664 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.052236 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.053551 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.053739 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:40.053819 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:40.054394 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 09:16:40.054530 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:16:40.055011 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:40.044941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:44413 (local address 127.17.16.65:47233)
0811 09:16:40.046750 (+ 1809us) negotiation.cc:107] Waiting for socket to connect
0811 09:16:40.046770 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:16:40.046904 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:16:40.047357 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:16:40.047362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:16:40.047469 (+ 107us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:16:40.048355 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:40.048368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.049835 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.049840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:40.052096 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:16:40.052109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.054027 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.054035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:16:40.054176 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:40.054212 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:16:40.054343 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 09:16:40.054446 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1651,"spinlock_wait_cycles":24320,"thread_start_us":1233,"threads_started":1}
I20260811 09:16:40.055891 4936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f9a1c76e164089bae502ac698146a7" candidate_uuid: "03183dc24eb84c1ea1853e788ac2f191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8952dc51d0204397bdc74cea948a5799" is_pre_election: true
I20260811 09:16:40.056155 4732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f9a1c76e164089bae502ac698146a7" candidate_uuid: "03183dc24eb84c1ea1853e788ac2f191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" is_pre_election: true
I20260811 09:16:40.056628 4936 raft_consensus.cc:2454] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03183dc24eb84c1ea1853e788ac2f191 in term 0.
I20260811 09:16:40.056777 4732 raft_consensus.cc:2454] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03183dc24eb84c1ea1853e788ac2f191 in term 0.
I20260811 09:16:40.057507 4424 leader_election.cc:302] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03183dc24eb84c1ea1853e788ac2f191, 8952dc51d0204397bdc74cea948a5799; no voters:
I20260811 09:16:40.058116 5046 raft_consensus.cc:2781] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:16:40.058319 5046 raft_consensus.cc:482] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:16:40.058506 5046 raft_consensus.cc:3037] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:16:40.061913 5046 raft_consensus.cc:504] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 } } peers { permanent_uuid: "03183dc24eb84c1ea1853e788ac2f191" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45229 } } peers { permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 } }
I20260811 09:16:40.062858 5046 leader_election.cc:288] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [CANDIDATE]: Term 1 election: Requested vote from peers 8952dc51d0204397bdc74cea948a5799 (127.17.16.67:38899), 97bfe6e3e29e4fdcbda8f28bbe238979 (127.17.16.66:44413)
I20260811 09:16:40.063230 4936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f9a1c76e164089bae502ac698146a7" candidate_uuid: "03183dc24eb84c1ea1853e788ac2f191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8952dc51d0204397bdc74cea948a5799"
I20260811 09:16:40.063498 4936 raft_consensus.cc:3037] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:16:40.063469 4732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50f9a1c76e164089bae502ac698146a7" candidate_uuid: "03183dc24eb84c1ea1853e788ac2f191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979"
I20260811 09:16:40.063735 4732 raft_consensus.cc:3037] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:16:40.066923 4732 raft_consensus.cc:2454] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03183dc24eb84c1ea1853e788ac2f191 in term 1.
I20260811 09:16:40.067214 4936 raft_consensus.cc:2454] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03183dc24eb84c1ea1853e788ac2f191 in term 1.
I20260811 09:16:40.067356 4421 leader_election.cc:302] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03183dc24eb84c1ea1853e788ac2f191, 97bfe6e3e29e4fdcbda8f28bbe238979; no voters:
I20260811 09:16:40.067852 5046 raft_consensus.cc:2781] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:16:40.068940 5046 raft_consensus.cc:686] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 1 LEADER]: Becoming Leader. State: Replica: 03183dc24eb84c1ea1853e788ac2f191, State: Running, Role: LEADER
I20260811 09:16:40.069666 5046 consensus_queue.cc:237] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 } } peers { permanent_uuid: "03183dc24eb84c1ea1853e788ac2f191" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45229 } } peers { permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 } }
I20260811 09:16:40.075173 4240 catalog_manager.cc:5495] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03183dc24eb84c1ea1853e788ac2f191 (127.17.16.65). New cstate: current_term: 1 leader_uuid: "03183dc24eb84c1ea1853e788ac2f191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03183dc24eb84c1ea1853e788ac2f191" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 } health_report { overall_health: UNKNOWN } } }
I20260811 09:16:40.135691 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:40.128597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45164 (local address 127.17.16.65:45229)
0811 09:16:40.128777 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:40.128782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:40.129076 (+ 294us) server_negotiation.cc:407] Connection header received
0811 09:16:40.129319 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:40.129322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:40.129402 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:40.129523 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:16:40.130093 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.130976 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.133030 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.133749 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.133842 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:40.134656 (+ 814us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:16:40.135140 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:16:40.135250 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:40.135363 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 09:16:40.135427 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:16:40.146158 4732 raft_consensus.cc:1261] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979 [term 1 FOLLOWER]: Refusing update from remote peer 03183dc24eb84c1ea1853e788ac2f191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:16:40.146242 4936 raft_consensus.cc:1261] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799 [term 1 FOLLOWER]: Refusing update from remote peer 03183dc24eb84c1ea1853e788ac2f191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:16:40.147290 5049 consensus_queue.cc:1035] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97bfe6e3e29e4fdcbda8f28bbe238979" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:16:40.147855 5046 consensus_queue.cc:1035] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8952dc51d0204397bdc74cea948a5799" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:16:40.161026 5056 mvcc.cc:205] Tried to move back new op lower bound from 7317257421387530240 to 7317257421089816576. Current Snapshot: MvccSnapshot[applied={T|T < 7317257421387530240}]
I20260811 09:16:40.162771 5058 mvcc.cc:205] Tried to move back new op lower bound from 7317257421387530240 to 7317257421089816576. Current Snapshot: MvccSnapshot[applied={T|T < 7317257421387530240}]
I20260811 09:16:40.180629 4239 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:16:40.181119 4239 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:16:40.185820 4239 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91398aefd574415fb211c5d077d43d8f: Sending DeleteTablet for 3 replicas of tablet 50f9a1c76e164089bae502ac698146a7
I20260811 09:16:40.187366 4903 tablet_service.cc:1514] Processing DeleteTablet for tablet 50f9a1c76e164089bae502ac698146a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:16:40 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:41996
I20260811 09:16:40.187958 4489 tablet_service.cc:1514] Processing DeleteTablet for tablet 50f9a1c76e164089bae502ac698146a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:16:40 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:45152
I20260811 09:16:40.192451 4703 tablet_service.cc:1514] Processing DeleteTablet for tablet 50f9a1c76e164089bae502ac698146a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:16:40 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:60286
I20260811 09:16:40.196103 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 4386
I20260811 09:16:40.221730 5074 tablet_replica.cc:330] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: stopping tablet replica
I20260811 09:16:40.222617 5074 raft_consensus.cc:2229] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:16:40.224762 5072 tablet_replica.cc:330] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: stopping tablet replica
I20260811 09:16:40.225179 5072 raft_consensus.cc:2229] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:16:40.225672 5072 raft_consensus.cc:2258] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:16:40.226413 5074 raft_consensus.cc:2258] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:16:40.227934 5075 tablet_replica.cc:330] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: stopping tablet replica
I20260811 09:16:40.228587 5075 raft_consensus.cc:2229] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:16:40.229130 5074 ts_tablet_manager.cc:1905] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:16:40.231066 5075 raft_consensus.cc:2258] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:16:40.237524 5074 ts_tablet_manager.cc:1918] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 11 09:16:40 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:16:40.237746 5074 log.cc:1196] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/50f9a1c76e164089bae502ac698146a7
I20260811 09:16:40.237879 5072 ts_tablet_manager.cc:1905] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:16:40.238488 5074 ts_tablet_manager.cc:1939] T 50f9a1c76e164089bae502ac698146a7 P 03183dc24eb84c1ea1853e788ac2f191: Deleting consensus metadata
I20260811 09:16:40.240701 4224 catalog_manager.cc:4841] TS 03183dc24eb84c1ea1853e788ac2f191 (127.17.16.65:45229): tablet 50f9a1c76e164089bae502ac698146a7 (table test-table [id=ad45aa37c9464341833daca863d7d1e5]) successfully deleted
I20260811 09:16:40.242678 5075 ts_tablet_manager.cc:1905] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:16:40.246320 5072 ts_tablet_manager.cc:1918] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:16:40.246599 5072 log.cc:1196] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/50f9a1c76e164089bae502ac698146a7
I20260811 09:16:40.247236 5072 ts_tablet_manager.cc:1939] T 50f9a1c76e164089bae502ac698146a7 P 8952dc51d0204397bdc74cea948a5799: Deleting consensus metadata
I20260811 09:16:40.249310 4222 catalog_manager.cc:4841] TS 8952dc51d0204397bdc74cea948a5799 (127.17.16.67:38899): tablet 50f9a1c76e164089bae502ac698146a7 (table test-table [id=ad45aa37c9464341833daca863d7d1e5]) successfully deleted
I20260811 09:16:40.249498 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:40.198585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.17.16.65:45229)
0811 09:16:40.198791 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:40.198796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:40.218262 (+ 19466us) server_negotiation.cc:407] Connection header received
0811 09:16:40.218878 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:40.218886 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:40.219017 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:40.219117 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:16:40.224176 (+ 5059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.225853 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.228638 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.229425 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.229544 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:40.239435 (+ 9891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:16:40.239455 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:16:40.239473 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:16:40.239521 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:16:40.244652 (+ 5131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:40.245711 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:40.245721 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:40.245724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:40.245969 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:40.246626 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:40.246632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:40.246634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:40.247464 (+ 830us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:16:40.247930 (+ 466us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:40.248626 (+ 696us) server_negotiation.cc:299] Negotiation successful
0811 09:16:40.248994 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:16:40.251305 4588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:16:40.251665 5075 ts_tablet_manager.cc:1918] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:16:40.252105 5075 log.cc:1196] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/50f9a1c76e164089bae502ac698146a7
I20260811 09:16:40.253120 5075 ts_tablet_manager.cc:1939] T 50f9a1c76e164089bae502ac698146a7 P 97bfe6e3e29e4fdcbda8f28bbe238979: Deleting consensus metadata
I20260811 09:16:40.254912 4217 catalog_manager.cc:4841] TS 97bfe6e3e29e4fdcbda8f28bbe238979 (127.17.16.66:44413): tablet 50f9a1c76e164089bae502ac698146a7 (table test-table [id=ad45aa37c9464341833daca863d7d1e5]) successfully deleted
I20260811 09:16:40.351270 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 4386
I20260811 09:16:40.375794 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 4606
Aug 11 09:16:40 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:16:40.391398 5032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:40.376531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.17.16.66:44413)
0811 09:16:40.376727 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:40.376733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:40.376756 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:16:40.376938 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:40.376943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:40.377007 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:40.377092 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:16:40.378808 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.379515 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.381345 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.381755 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.381865 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:40.386138 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:16:40.386153 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:16:40.386168 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:16:40.386205 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:16:40.388974 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:40.389673 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:40.389680 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:40.389682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:40.389805 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:40.390084 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:40.390087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:40.390089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:40.390447 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:16:40.390666 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:40.390864 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 09:16:40.391049 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:16:40.392079 4796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:16:40.480108 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 4606
I20260811 09:16:40.503429 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 4814
Aug 11 09:16:40 dist-test-slave-ncp9 krb5kdc[3463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:16:40.518006 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:16:40.504203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.17.16.67:38899)
0811 09:16:40.504385 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:16:40.504390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:16:40.504409 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:16:40.504556 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:16:40.504560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:16:40.504645 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:16:40.504729 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:16:40.506104 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.506847 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.508088 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:16:40.508487 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:16:40.508575 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:16:40.512766 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:16:40.512783 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:16:40.512801 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:16:40.512836 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:16:40.515381 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:40.516075 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:40.516082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:40.516085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:40.516214 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:16:40.516501 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:16:40.516504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:16:40.516506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:16:40.516864 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:16:40.517312 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0811 09:16:40.517493 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 09:16:40.517685 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:16:40.518669 4990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:16:40.606951 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 4814
I20260811 09:16:40.632547 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 4186
I20260811 09:16:40.633811 4291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:16:40.746896 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 4186
I20260811 09:16:40.780949 17473 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 09:16:41.125226 17473 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 09:16:41.125499 17473 mini_ranger.cc:66] Stopping Ranger...
I20260811 09:16:41.468981 17473 mini_ranger.cc:68] Stopped Ranger
2026-08-11 09:16:41.469 UTC [3513] LOG: received smart shutdown request
2026-08-11 09:16:41.472 UTC [3513] LOG: background worker "logical replication launcher" (PID 3526) exited with exit code 1
2026-08-11 09:16:41.473 UTC [3518] LOG: shutting down
2026-08-11 09:16:41.612 UTC [3513] LOG: database system is shut down
2026-08-11T09:16:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50984 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:16:41 dist-test-slave-ncp9 krb5kdc[5103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:16:41 dist-test-slave-ncp9 krb5kdc[5103](info): set up 2 sockets
Aug 11 09:16:41 dist-test-slave-ncp9 krb5kdc[5103](info): commencing operation
krb5kdc: starting...
W20260811 09:16:42.979418 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:16:43 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:16:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:16:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/rangeradmin_127.17.16.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/rangeradmin_127.17.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/rangerlookup_127.17.16.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/rangerlookup_127.17.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/HTTP_127.17.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/HTTP_127.17.16.84@KRBTEST.COM.keytab.
I20260811 09:16:43.329195 17473 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskNGP8Lb/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/postgres -l logfile start
2026-08-11 09:16:45.606 UTC [5149] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 09:16:45.606 UTC [5149] LOG: listening on IPv4 address "127.17.16.84", port 49475
2026-08-11 09:16:45.609 UTC [5149] LOG: listening on Unix socket "/tmp/.s.PGSQL.49475"
2026-08-11 09:16:45.631 UTC [5156] LOG: database system was shut down at 2026-08-11 09:16:44 UTC
2026-08-11 09:16:45.634 UTC [5149] LOG: database system is ready to accept connections
I20260811 09:16:46.730036 17473 mini_postgres.cc:96] Postgres bound to 49475
2026-08-11 09:16:46.752 UTC [5168] FATAL: database "slave" does not exist
127.17.16.84:49475 - accepting connections
I20260811 09:16:46.752867 17473 mini_ranger.cc:161] Starting Ranger...
I20260811 09:16:46.780596 17473 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 09:16:47.215874 17473 mini_ranger.cc:87] Created ranger Postgres database
I20260811 09:16:47.215986 17473 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-admin
2026-08-11 09:16:47,521 [I] DB FLAVOR :POSTGRES
2026-08-11 09:16:47,521 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:16:47,522 [I] Checking connection..
2026-08-11 09:16:47,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 09:16:47,779 [I] Checking connection passed.
2026-08-11 09:16:47,779 [I] --------- Verifying version history table ---------
2026-08-11 09:16:47,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:16:48,026 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:16:48,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:16:48,254 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:16:48,254 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 09:16:48,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 09:16:48.497 UTC [5266] WARNING: there is no transaction in progress
2026-08-11 09:16:48,506 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 09:16:48,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:16:48,759 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 09:16:48,759 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:16:48,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 09:16:49,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 09:16:49,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ncp9.c.gcp-upstream.internal','N') ;"
2026-08-11 09:16:49,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:16:49,758 [I] Table x_portal_user does not exist in database ranger
2026-08-11 09:16:49,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:16:49,999 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 09:16:49,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:16:50,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:16:50,483 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 09:16:50,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 09:16:51.373 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.387 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.402 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.416 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.430 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.448 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.513 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.519 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.532 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.542 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.551 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.564 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.573 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.579 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.585 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:51.591 UTC [5455] WARNING: there is no transaction in progress
2026-08-11 09:16:52,168 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 09:16:52,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 09:16:52,416 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 09:16:52,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:16:52,676 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 09:16:52,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:16:52,921 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 09:16:52,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:16:53,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:16:53,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ncp9.c.gcp-upstream.internal';"
2026-08-11 09:16:53,674 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 09:16:53,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 09:16:53,921 [I] DB_PATCHES have already been applied
I20260811 09:16:53.926895 17473 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-admin:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskNGP8Lb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 09:16:53.926977 17473 mini_ranger.cc:193] Using host: 127.17.16.84
I20260811 09:16:53.946110 17473 mini_ranger.cc:227] Ranger admin URL: http://127.17.16.84:41471
Aug 11, 2026 9:16:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:16:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:16:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 9:16:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 9:16:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 9:16:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 9:16:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 09:16:55.179280 17473 mini_ranger.cc:160] Time spent starting Ranger: real 8.426s user 0.003s sys 0.052s
Aug 11, 2026 9:17:13 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 9:17:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 9:17:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 9:17:13 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 9:17:14 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 09:17:15.510919 17473 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.16.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.16.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/rangerkms_127.17.16.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.16.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/rangerkms_127.17.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.16.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.16.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.16.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/HTTP_127.17.16.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.16.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/HTTP_127.17.16.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 09:17:15 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439835, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 09:17:15.699723 17473 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 09:17:15.723204 17473 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 09:17:16.506196 17473 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 09:17:16.506311 17473 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-kms
I20260811 09:17:16.506348 17473 mini_ranger_kms.cc:227] Using postgres at 127.17.16.84:49475
2026-08-11 09:17:16,846 [I] DB FLAVOR :POSTGRES
2026-08-11 09:17:16,846 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:17:16,846 [I] Checking connection
2026-08-11 09:17:16,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 09:17:17,090 [I] connection success
2026-08-11 09:17:17,090 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 09:17:17,090 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 09:17:17,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 09:17:17,326 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 09:17:17,326 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:17:17,326 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 09:17:17,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.16.84:49475/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 09:17:17,602 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 09:17:18.014780 17473 mini_ranger_kms.cc:310] Created kms service
I20260811 09:17:18.221225 17473 mini_ranger_kms.cc:326] Created kudu user
I20260811 09:17:18.263336 17473 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 09:17:18 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439835, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:17:18.648579 17473 mini_ranger_kms.cc:384] Added ranger policy
I20260811 09:17:18.648918 17473 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-kms:/tmp/dist-test-taskNGP8Lb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskNGP8Lb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 09:17:18.648988 17473 mini_ranger_kms.cc:240] Using host: 127.17.16.84
I20260811 09:17:18.669588 17473 mini_ranger_kms.cc:276] Ranger KMS PID: 5750
I20260811 09:17:18.669699 17473 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.16.84:50239
Aug 11, 2026 9:17:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:17:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:17:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 9:17:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 9:17:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 9:17:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 09:17:20 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439840, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.16.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:21.285949 17473 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.586s user 0.072s sys 0.299s
I20260811 09:17:21.286221 17473 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 09:17:21.286415 17473 mini_ranger_kms.cc:401] 127.17.16.84:50239/kms/v1/keys
Aug 11 09:17:21 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439840, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.16.84@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
Aug 11 09:17:21 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439840, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.16.84@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
Aug 11 09:17:23 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:17:24.058667 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41767
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41767
--ranger_config_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:50239/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:17:24.388075 5848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:24.388346 5848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:24.388465 5848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:24.395922 5848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:17:24.396031 5848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:24.396087 5848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:24.396165 5848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:17:24.396227 5848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:17:24.406165 5848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.16.84:50239/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41767
--ranger_config_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:24.407742 5848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:24.409083 5848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:24.421561 5856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:24.421563 5860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:24.421749 5862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:24.422168 5848 server_base.cc:1034] running on GCE node
I20260811 09:17:24.423296 5848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:24.424911 5848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:24.426163 5848 hybrid_clock.cc:648] HybridClock initialized: now 1786439844426114 us; error 31 us; skew 500 ppm
I20260811 09:17:24.426537 5848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:24 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439844, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:24.430465 5848 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:17:24.432080 5848 webserver.cc:460] Webserver started at http://127.17.16.126:42325/ using document root <none> and password file <none>
I20260811 09:17:24.432842 5848 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:24.433001 5848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:24.433341 5848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:17:24 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439844, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:17:24.481791 5848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "e0ce0d96016645f491207306d7f93eaf"
format_stamp: "Formatted at 2026-08-11 09:17:24 on dist-test-slave-ncp9"
server_key: "bdf7f93bfea39c3ebf58af0b661f7665"
server_key_iv: "d5e6433cfc2e4f2afe871f1dd32b4ba7"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:24.482697 5848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "e0ce0d96016645f491207306d7f93eaf"
format_stamp: "Formatted at 2026-08-11 09:17:24 on dist-test-slave-ncp9"
server_key: "bdf7f93bfea39c3ebf58af0b661f7665"
server_key_iv: "d5e6433cfc2e4f2afe871f1dd32b4ba7"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:24.488111 5848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:17:24.503856 5876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:24.505386 5848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:17:24.505574 5848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "e0ce0d96016645f491207306d7f93eaf"
format_stamp: "Formatted at 2026-08-11 09:17:24 on dist-test-slave-ncp9"
server_key: "bdf7f93bfea39c3ebf58af0b661f7665"
server_key_iv: "d5e6433cfc2e4f2afe871f1dd32b4ba7"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:24.505786 5848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:24.525426 5848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:24.529264 5848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:24.529565 5848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:24.548033 5848 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:41767
I20260811 09:17:24.548054 5963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:41767 every 8 connection(s)
I20260811 09:17:24.549719 5848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:17:24.555208 5965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:24.555367 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 5848
I20260811 09:17:24.555562 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:17:24.556255 17473 external_mini_cluster.cc:1428] Setting key 97ddd311d489b614957285214c355c4f
I20260811 09:17:24.570035 5965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: Bootstrap starting.
I20260811 09:17:24.575380 5965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:24.576779 5965 log.cc:824] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:24.580195 5965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: No bootstrap required, opened a new log
Aug 11 09:17:24 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:24.590400 5965 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } }
I20260811 09:17:24.590817 5965 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:24.590929 5965 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0ce0d96016645f491207306d7f93eaf, State: Initialized, Role: FOLLOWER
I20260811 09:17:24.591507 5965 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } }
I20260811 09:17:24.591720 5965 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:17:24.591717 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:24.558306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.17.16.126:41767)
0811 09:17:24.573285 (+ 14979us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:24.573300 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:24.573348 (+ 48us) server_negotiation.cc:407] Connection header received
0811 09:17:24.575107 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:24.575131 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:24.575516 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:24.575928 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:24.577457 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:24.579026 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:24.580302 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:24.581151 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:24.581314 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:24.584876 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:24.584883 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:24.584922 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:24.584967 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:24.587781 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:24.588410 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:24.588416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:24.588422 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:24.588583 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:24.588966 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:24.588971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:24.588974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:24.589497 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:24.589789 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:24.590211 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 09:17:24.590586 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14762,"thread_start_us":14529,"threads_started":1}
I20260811 09:17:24.591862 5965 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:17:24.592087 5965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:24.594794 5965 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } }
I20260811 09:17:24.595242 5965 leader_election.cc:302] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0ce0d96016645f491207306d7f93eaf; no voters:
I20260811 09:17:24.595727 5965 leader_election.cc:288] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:17:24.595839 5972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:17:24.596473 5972 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 1 LEADER]: Becoming Leader. State: Replica: e0ce0d96016645f491207306d7f93eaf, State: Running, Role: LEADER
I20260811 09:17:24.597015 5972 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } }
I20260811 09:17:24.597854 5965 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:17:24.599931 5976 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0ce0d96016645f491207306d7f93eaf. Latest consensus state: current_term: 1 leader_uuid: "e0ce0d96016645f491207306d7f93eaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } } }
I20260811 09:17:24.600365 5976 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:24.599952 5973 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0ce0d96016645f491207306d7f93eaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } } }
I20260811 09:17:24.600811 5973 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:24.608657 5983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:17:24.613031 5983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:17:24.613793 5965 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 09:17:24.633419 5983 catalog_manager.cc:1347] Generated new cluster ID: 2818454f047947e9ab82756f9aa11e60
I20260811 09:17:24.633574 5983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:17:24.657279 5983 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:17:24.659251 5983 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:17:24.668409 5983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: Generated new TSK 0
I20260811 09:17:24.669432 5983 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:17:25.371706 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:50239/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41767
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:17:25 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439845, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 09:17:25.808558 6038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:25.808920 6038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:25.809101 6038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:25.816861 6038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:25.817018 6038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:25.817194 6038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:17:25.827323 6038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.16.84:50239/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:25.828603 6038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:25.829886 6038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:25.844241 6059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:25.845059 6061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:25.853293 6063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:25.853823 6038 server_base.cc:1034] running on GCE node
I20260811 09:17:25.854446 6038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:25.856528 6038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:25.857862 6038 hybrid_clock.cc:648] HybridClock initialized: now 1786439845857761 us; error 89 us; skew 500 ppm
I20260811 09:17:25.858302 6038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:25 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439845, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:25.861714 6038 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:17:25.863565 6038 webserver.cc:460] Webserver started at http://127.17.16.65:42497/ using document root <none> and password file <none>
I20260811 09:17:25.864457 6038 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:25.864607 6038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:25.864977 6038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:17:25 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439845, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:17:25.889720 6038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "d3815f99550d4276896e88056ddf0c06"
format_stamp: "Formatted at 2026-08-11 09:17:25 on dist-test-slave-ncp9"
server_key: "74ea9b935b376888e236cf8cc9395692"
server_key_iv: "828ecfada7acbd925424073353b9606c"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:25.890900 6038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "d3815f99550d4276896e88056ddf0c06"
format_stamp: "Formatted at 2026-08-11 09:17:25 on dist-test-slave-ncp9"
server_key: "74ea9b935b376888e236cf8cc9395692"
server_key_iv: "828ecfada7acbd925424073353b9606c"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:25.904660 6038 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.004s
I20260811 09:17:25.937568 6075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:25.939599 6038 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.005s sys 0.011s
I20260811 09:17:25.939900 6038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "d3815f99550d4276896e88056ddf0c06"
format_stamp: "Formatted at 2026-08-11 09:17:25 on dist-test-slave-ncp9"
server_key: "74ea9b935b376888e236cf8cc9395692"
server_key_iv: "828ecfada7acbd925424073353b9606c"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:25.940304 6038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:25.992959 6038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:25.997898 6038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:25.998217 6038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:25.999228 6038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:26.000730 6038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:26.000806 6038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:26.000922 6038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:26.000979 6038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 09:17:26.068912 6038 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:35127
I20260811 09:17:26.068920 6264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:35127 every 8 connection(s)
I20260811 09:17:26.070360 6038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:17:26.074482 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 6038
I20260811 09:17:26.074683 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:17:26.075511 17473 external_mini_cluster.cc:1428] Setting key 5ec0b1b9711d42a2c81ce5a6e3137cb8
Aug 11 09:17:26 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439845, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:26.106393 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:26.074362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41767 (local address 127.17.16.65:51201)
0811 09:17:26.075257 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:26.075307 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:26.078369 (+ 3062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:26.079320 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:26.079339 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:26.079974 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:26.080790 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:26.080816 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:26.082421 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:26.082427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:26.084465 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:26.084481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:26.085770 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:26.085778 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:26.086080 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:26.086884 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:26.086915 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:26.092747 (+ 5832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:26.100015 (+ 7268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:26.100026 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:26.100046 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:26.100741 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:26.101636 (+ 895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:26.101642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:26.101646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:26.101925 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:26.103604 (+ 1679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:26.103614 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:26.104072 (+ 458us) client_negotiation.cc:769] Sending connection context
0811 09:17:26.104365 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 09:17:26.104948 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":1009,"threads_started":1}
I20260811 09:17:26.108093 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:26.075276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:51201 (local address 127.17.16.126:41767)
0811 09:17:26.076760 (+ 1484us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:26.076767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:26.076786 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:17:26.078678 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:26.078683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:26.078786 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:26.078887 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:26.081040 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:26.082207 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:26.084637 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:26.085309 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:26.085410 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:26.093050 (+ 7640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:26.093058 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:26.093063 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:26.093105 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:26.099857 (+ 6752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:26.100948 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:26.100954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:26.100958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:26.101489 (+ 531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:26.103070 (+ 1581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:26.103076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:26.103080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:26.103444 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:26.107329 (+ 3885us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:26.107462 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 09:17:26.107702 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1378,"thread_start_us":1191,"threads_started":1}
I20260811 09:17:26.109935 6265 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41767
I20260811 09:17:26.110275 6265 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:26.111159 6265 heartbeater.cc:511] Master 127.17.16.126:41767 requested a full tablet report, sending...
I20260811 09:17:26.113822 5903 ts_manager.cc:194] Registered new tserver with Master: d3815f99550d4276896e88056ddf0c06 (127.17.16.65:35127)
I20260811 09:17:26.115944 5903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:51201
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:17:26.160526 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:50239/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41767
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:17:26 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439845, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
Aug 11 09:17:26 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439845, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
W20260811 09:17:26.654400 6274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:26.654917 6274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:26.655158 6274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:26.666780 6274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:26.666982 6274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:26.667240 6274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:17:26.682646 6274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.16.84:50239/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:26.684616 6274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:26.686439 6274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:26.703311 6286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:26.704921 6288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:26.712234 6290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:26.714274 6274 server_base.cc:1034] running on GCE node
I20260811 09:17:26.715103 6274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:26.716452 6274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:26.717916 6274 hybrid_clock.cc:648] HybridClock initialized: now 1786439846717848 us; error 60 us; skew 500 ppm
I20260811 09:17:26.718334 6274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:26 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439846, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:26.722692 6274 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:17:26.736733 6274 webserver.cc:460] Webserver started at http://127.17.16.66:42987/ using document root <none> and password file <none>
I20260811 09:17:26.737818 6274 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:26.737969 6274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:26.738372 6274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:17:26 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439846, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:17:26.756608 6274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "f58610f4c6a647db8d2c69ae9f40e05b"
format_stamp: "Formatted at 2026-08-11 09:17:26 on dist-test-slave-ncp9"
server_key: "31cf032c1fa32b2ec829eedbf762a0c4"
server_key_iv: "8497d79a4a4f39da636d89642ccd7266"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:26.757478 6274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "f58610f4c6a647db8d2c69ae9f40e05b"
format_stamp: "Formatted at 2026-08-11 09:17:26 on dist-test-slave-ncp9"
server_key: "31cf032c1fa32b2ec829eedbf762a0c4"
server_key_iv: "8497d79a4a4f39da636d89642ccd7266"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:26.762964 6274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:17:26.779888 6309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:26.781423 6274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:17:26.781584 6274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "f58610f4c6a647db8d2c69ae9f40e05b"
format_stamp: "Formatted at 2026-08-11 09:17:26 on dist-test-slave-ncp9"
server_key: "31cf032c1fa32b2ec829eedbf762a0c4"
server_key_iv: "8497d79a4a4f39da636d89642ccd7266"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:26.781792 6274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:26.812266 6274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:26.815851 6274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:26.816160 6274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:26.817121 6274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:26.818552 6274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:26.818634 6274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:26.818764 6274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:26.818822 6274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:26.850791 6274 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:42009
I20260811 09:17:26.850827 6504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:42009 every 8 connection(s)
I20260811 09:17:26.851997 6274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:17:26.858170 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 6274
I20260811 09:17:26.858331 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:17:26.858774 17473 external_mini_cluster.cc:1428] Setting key 1be5290635890104e203c4f1dd488aee
Aug 11 09:17:26 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439846, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:26.871305 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:26.854396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:33295 (local address 127.17.16.126:41767)
0811 09:17:26.854867 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:26.854874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:26.855772 (+ 898us) server_negotiation.cc:407] Connection header received
0811 09:17:26.857630 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:26.857634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:26.857719 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:26.857816 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:26.859271 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:26.860047 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:26.861817 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:26.862404 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:26.862557 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:26.865838 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:26.865843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:26.865846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:26.865879 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:26.868244 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:26.869047 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:26.869050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:26.869052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:26.869181 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:26.869622 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:26.869625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:26.869626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:26.869931 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:26.870054 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:26.870772 (+ 718us) server_negotiation.cc:299] Negotiation successful
0811 09:17:26.870966 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":217,"threads_started":1}
I20260811 09:17:26.872383 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:26.854900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41767 (local address 127.17.16.66:33295)
0811 09:17:26.855614 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:26.855660 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:26.857384 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:26.858004 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:26.858011 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:26.858482 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:26.859096 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:26.859114 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:26.860203 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:26.860208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:26.861674 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:26.861687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:26.862593 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:26.862598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:26.862801 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:26.863447 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:26.863470 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:26.865624 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:26.868381 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:26.868389 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:26.868403 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:26.868950 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:26.869273 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:26.869278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:26.869281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:26.869535 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:26.870047 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:26.870054 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:26.870414 (+ 360us) client_negotiation.cc:769] Sending connection context
0811 09:17:26.870694 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 09:17:26.871152 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":286,"threads_started":1}
I20260811 09:17:26.873683 6506 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41767
I20260811 09:17:26.873948 6506 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:26.874590 6506 heartbeater.cc:511] Master 127.17.16.126:41767 requested a full tablet report, sending...
I20260811 09:17:26.876032 5903 ts_manager.cc:194] Registered new tserver with Master: f58610f4c6a647db8d2c69ae9f40e05b (127.17.16.66:42009)
I20260811 09:17:26.877174 5903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:33295
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:17:26.919902 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:50239/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41767
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:17:27.119647 6265 heartbeater.cc:503] Master 127.17.16.126:41767 was elected leader, sending a full tablet report...
W20260811 09:17:27.262526 6518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:27.262801 6518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:27.262935 6518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:27.269910 6518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:27.270026 6518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:27.270176 6518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:17:27.279636 6518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.16.84:50239/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:27.280925 6518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:27.282215 6518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:27.294601 6536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:27.294660 6530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:27.294629 6533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:27.295301 6518 server_base.cc:1034] running on GCE node
I20260811 09:17:27.295862 6518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:27.296751 6518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:27.297959 6518 hybrid_clock.cc:648] HybridClock initialized: now 1786439847297902 us; error 53 us; skew 500 ppm
I20260811 09:17:27.298278 6518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:27 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439847, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:27.301345 6518 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:17:27.302780 6518 webserver.cc:460] Webserver started at http://127.17.16.67:42531/ using document root <none> and password file <none>
I20260811 09:17:27.303496 6518 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:27.303617 6518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:27.303910 6518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:17:27 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439847, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:17:27.320050 6518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "c3919a65646d48b9b73d0e7d4c63528b"
format_stamp: "Formatted at 2026-08-11 09:17:27 on dist-test-slave-ncp9"
server_key: "8f8c96e6f5fc7ffc5e8fca2414e0573b"
server_key_iv: "ada43d47fe05eac7c70df109aa20bc89"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:27.320851 6518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "c3919a65646d48b9b73d0e7d4c63528b"
format_stamp: "Formatted at 2026-08-11 09:17:27 on dist-test-slave-ncp9"
server_key: "8f8c96e6f5fc7ffc5e8fca2414e0573b"
server_key_iv: "ada43d47fe05eac7c70df109aa20bc89"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:27.326323 6518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:17:27.342310 6549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:27.343745 6518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:17:27.343899 6518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "c3919a65646d48b9b73d0e7d4c63528b"
format_stamp: "Formatted at 2026-08-11 09:17:27 on dist-test-slave-ncp9"
server_key: "8f8c96e6f5fc7ffc5e8fca2414e0573b"
server_key_iv: "ada43d47fe05eac7c70df109aa20bc89"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:27.344113 6518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:27.375798 6518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:27.379627 6518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:27.379911 6518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:27.380884 6518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:27.382333 6518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:27.382408 6518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:27.382511 6518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:27.382560 6518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:27.423453 6518 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:32885
I20260811 09:17:27.423530 6729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:32885 every 8 connection(s)
I20260811 09:17:27.424758 6518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:17:27.425100 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 6518
I20260811 09:17:27.425293 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:17:27.425947 17473 external_mini_cluster.cc:1428] Setting key a5a6bcccdfd655d674a5e00e3eca7d11
Aug 11 09:17:27 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439847, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:27.451104 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:27.428087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41767 (local address 127.17.16.67:34411)
0811 09:17:27.430501 (+ 2414us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:27.430537 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:27.432079 (+ 1542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:27.433108 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:27.433120 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:27.433671 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:27.434399 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:27.434423 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:27.435416 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:27.435421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:27.437290 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:27.437304 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:27.438026 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:27.438032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:27.438295 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:27.438977 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:27.439006 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:27.441931 (+ 2925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:27.445631 (+ 3700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:27.445641 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:27.445656 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:27.446235 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:27.447013 (+ 778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:27.447018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:27.447021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:27.447296 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:27.448493 (+ 1197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:27.448501 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:27.448884 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 09:17:27.449222 (+ 338us) client_negotiation.cc:240] Negotiation successful
0811 09:17:27.449719 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2170,"thread_start_us":1781,"threads_started":1}
I20260811 09:17:27.452245 6734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:27.428679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:34411 (local address 127.17.16.126:41767)
0811 09:17:27.432576 (+ 3897us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:27.432581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:27.432594 (+ 13us) server_negotiation.cc:407] Connection header received
0811 09:17:27.432643 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:27.432647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:27.432716 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:27.432794 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:27.434561 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:27.435277 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:27.437415 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:27.437824 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:27.437912 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:27.442144 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:27.442149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:27.442152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:27.442183 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:27.445474 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:27.446375 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:27.446380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:27.446384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:27.446549 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:27.447441 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:27.447446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:27.447449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:27.448277 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:27.451496 (+ 3219us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:27.451638 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 09:17:27.451870 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3804,"thread_start_us":3707,"threads_started":1}
I20260811 09:17:27.453624 6730 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41767
I20260811 09:17:27.453925 6730 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:27.454716 6730 heartbeater.cc:511] Master 127.17.16.126:41767 requested a full tablet report, sending...
I20260811 09:17:27.456284 5901 ts_manager.cc:194] Registered new tserver with Master: c3919a65646d48b9b73d0e7d4c63528b (127.17.16.67:32885)
I20260811 09:17:27.457527 5901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:34411
I20260811 09:17:27.467054 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:17:27.468655 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 6038
Aug 11 09:17:27 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:17:27.486378 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:27.469513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41764 (local address 127.17.16.65:35127)
0811 09:17:27.470339 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:27.470348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:27.470378 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:17:27.470451 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:27.470460 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:27.470709 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:27.470870 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:27.472268 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:27.473836 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:27.475217 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:27.475988 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:27.476126 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:27.480160 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:27.480174 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:27.480194 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:27.480235 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:27.483635 (+ 3400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:27.484244 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:27.484253 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:27.484257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:27.484427 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:27.484750 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:27.484754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:27.484758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:27.485205 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:27.485519 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:27.485705 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 09:17:27.485982 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":488,"threads_started":1}
I20260811 09:17:27.487810 6258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:27.618474 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 6038
I20260811 09:17:27.651927 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 6274
Aug 11 09:17:27 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:17:27.678107 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:27.653358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55662 (local address 127.17.16.66:42009)
0811 09:17:27.657539 (+ 4181us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:27.657550 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:27.657588 (+ 38us) server_negotiation.cc:407] Connection header received
0811 09:17:27.657677 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:27.657687 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:27.657990 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:27.658174 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:27.659775 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:27.661484 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:27.663570 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:27.664482 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:27.664624 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:27.670347 (+ 5723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:27.670363 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:27.670384 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:27.670435 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:27.674434 (+ 3999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:27.675333 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:27.675343 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:27.675346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:27.675547 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:27.675964 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:27.675969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:27.675972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:27.676550 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:27.676881 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:27.677367 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 09:17:27.677681 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3949,"thread_start_us":3805,"threads_started":1}
I20260811 09:17:27.679703 6499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:27.767514 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 6274
I20260811 09:17:27.789891 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 6518
Aug 11 09:17:27 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:17:27.806174 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:27.790914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36964 (local address 127.17.16.67:32885)
0811 09:17:27.791559 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:27.791565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:27.791596 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:17:27.791657 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:27.791664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:27.791873 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:27.792013 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:27.793378 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:27.794400 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:27.795658 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:27.796207 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:27.796328 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:27.800720 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:27.800732 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:27.800747 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:27.800780 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:27.803326 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:27.803965 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:27.803973 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:27.803975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:27.804105 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:27.804442 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:27.804446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:27.804448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:27.804807 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:27.805186 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:27.805566 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 09:17:27.805837 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":307,"threads_started":1}
I20260811 09:17:27.807397 6724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:27.888926 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 6518
I20260811 09:17:27.909271 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 5848
I20260811 09:17:27.910524 5957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:28.014777 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 5848
I20260811 09:17:28.045059 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41767
--webserver_interface=127.17.16.126
--webserver_port=42325
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41767
--ranger_config_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:50239/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.16.84:50239/kms with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:17:28.371938 6745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:28.372233 6745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:28.372363 6745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:28.379429 6745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:17:28.379544 6745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:28.379619 6745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:28.379694 6745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:17:28.379755 6745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:17:28.389529 6745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.16.84:50239/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41767
--ranger_config_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=42325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:28.390756 6745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:28.392012 6745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:28.403044 6756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:28.403224 6753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:28.403215 6751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:28.404073 6745 server_base.cc:1034] running on GCE node
I20260811 09:17:28.404734 6745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:28.406077 6745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:28.407269 6745 hybrid_clock.cc:648] HybridClock initialized: now 1786439848407233 us; error 29 us; skew 500 ppm
I20260811 09:17:28.407562 6745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:28 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439848, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:28.410593 6745 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:17:28.411913 6745 webserver.cc:460] Webserver started at http://127.17.16.126:42325/ using document root <none> and password file <none>
I20260811 09:17:28.412528 6745 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:28.412645 6745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:17:28 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439848, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:17:28.490576 6745 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:17:28.493695 6768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:28.495182 6745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:17:28.495353 6745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "e0ce0d96016645f491207306d7f93eaf"
format_stamp: "Formatted at 2026-08-11 09:17:24 on dist-test-slave-ncp9"
server_key: "bdf7f93bfea39c3ebf58af0b661f7665"
server_key_iv: "d5e6433cfc2e4f2afe871f1dd32b4ba7"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:28.496310 6745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:28.544683 6745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:28.548218 6745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:28.548480 6745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:28.565326 6745 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:41767
I20260811 09:17:28.565366 6843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:41767 every 8 connection(s)
I20260811 09:17:28.566742 6745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:17:28.572487 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 6745
I20260811 09:17:28.573369 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:50239/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:35127
--local_ip_for_outbound_sockets=127.17.16.65
--tserver_master_addrs=127.17.16.126:41767
--webserver_port=42497
--webserver_interface=127.17.16.65
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.16.84:50239/kms with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:17:28.574414 6845 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:17:28.577538 6845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: Bootstrap starting.
I20260811 09:17:28.595486 6845 log.cc:824] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:28.604290 6845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:17:28.604772 6845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: Bootstrap complete.
I20260811 09:17:28.613237 6845 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } }
I20260811 09:17:28.614061 6845 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0ce0d96016645f491207306d7f93eaf, State: Initialized, Role: FOLLOWER
I20260811 09:17:28.614627 6845 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } }
I20260811 09:17:28.614915 6845 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:17:28.615027 6845 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:17:28.615181 6845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:17:28.618754 6845 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } }
I20260811 09:17:28.619191 6845 leader_election.cc:302] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0ce0d96016645f491207306d7f93eaf; no voters:
I20260811 09:17:28.619725 6845 leader_election.cc:288] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:17:28.619905 6850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:17:28.620819 6850 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [term 2 LEADER]: Becoming Leader. State: Replica: e0ce0d96016645f491207306d7f93eaf, State: Running, Role: LEADER
I20260811 09:17:28.621429 6850 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } }
I20260811 09:17:28.621600 6845 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:17:28.623900 6853 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0ce0d96016645f491207306d7f93eaf. Latest consensus state: current_term: 2 leader_uuid: "e0ce0d96016645f491207306d7f93eaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } } }
I20260811 09:17:28.624281 6853 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:28.624742 6852 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e0ce0d96016645f491207306d7f93eaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ce0d96016645f491207306d7f93eaf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41767 } } }
I20260811 09:17:28.625056 6852 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:28.631659 6862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:17:28.637665 6862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:17:28.639093 6862 catalog_manager.cc:1259] Loaded cluster ID: 2818454f047947e9ab82756f9aa11e60
I20260811 09:17:28.639258 6862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:17:28.643314 6862 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:17:28.644807 6862 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: Loaded TSK: 0
I20260811 09:17:28.646417 6862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:17:28.649215 6845 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 09:17:29.023535 6848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:29.023972 6848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:29.024197 6848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:29.035502 6848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:29.035648 6848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:29.035842 6848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:17:29.051389 6848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.16.84:50239/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:35127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=42497
--tserver_master_addrs=127.17.16.126:41767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:29.053414 6848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:29.055310 6848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:29.088629 6902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:29.089807 6900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:29.096983 6905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:29.097245 6848 server_base.cc:1034] running on GCE node
I20260811 09:17:29.098121 6848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:29.102288 6848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:29.105005 6848 hybrid_clock.cc:648] HybridClock initialized: now 1786439849104916 us; error 68 us; skew 500 ppm
I20260811 09:17:29.105475 6848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:29 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:29.110208 6848 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:17:29.145351 6848 webserver.cc:460] Webserver started at http://127.17.16.65:42497/ using document root <none> and password file <none>
I20260811 09:17:29.146507 6848 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:29.146791 6848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:17:29 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:17:29.182806 6848 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 09:17:29.189647 6917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:29.191510 6848 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.006s
I20260811 09:17:29.191690 6848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "d3815f99550d4276896e88056ddf0c06"
format_stamp: "Formatted at 2026-08-11 09:17:25 on dist-test-slave-ncp9"
server_key: "74ea9b935b376888e236cf8cc9395692"
server_key_iv: "828ecfada7acbd925424073353b9606c"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:29.193114 6848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:29.218338 6848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:29.223762 6848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:29.224123 6848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:29.225412 6848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:29.233395 6848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:29.233569 6848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 09:17:29.233744 6848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:29.233875 6848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:29.278803 6848 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:35127
I20260811 09:17:29.278875 7096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:35127 every 8 connection(s)
I20260811 09:17:29.280586 6848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:17:29.284687 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 6848
I20260811 09:17:29.285550 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:50239/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:42009
--local_ip_for_outbound_sockets=127.17.16.66
--tserver_master_addrs=127.17.16.126:41767
--webserver_port=42987
--webserver_interface=127.17.16.66
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.16.84:50239/kms with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:17:29 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:29.338747 7099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:29.283715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:39903 (local address 127.17.16.126:41767)
0811 09:17:29.295186 (+ 11471us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:29.295206 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:29.304988 (+ 9782us) server_negotiation.cc:407] Connection header received
0811 09:17:29.307281 (+ 2293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:29.307309 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:29.307850 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:29.308243 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:29.310676 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:29.312618 (+ 1942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:29.315024 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:29.316034 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:29.316197 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:29.327840 (+ 11643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:29.327851 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:29.327871 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:29.327919 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:29.331759 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:29.332723 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:29.332733 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:29.332742 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:29.333033 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:29.333603 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:29.333608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:29.333612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:29.334239 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:29.334433 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:29.337205 (+ 2772us) server_negotiation.cc:299] Negotiation successful
0811 09:17:29.337612 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11158,"thread_start_us":11793,"threads_started":1}
I20260811 09:17:29.339622 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:29.283759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41767 (local address 127.17.16.65:39903)
0811 09:17:29.296663 (+ 12904us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:29.296722 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:29.306486 (+ 9764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:29.309031 (+ 2545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:29.309046 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:29.309673 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:29.310420 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:29.310448 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:29.312858 (+ 2410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:29.312864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:29.314840 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:29.314868 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:29.316661 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:29.316670 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:29.316982 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:29.322919 (+ 5937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:29.322950 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:29.326046 (+ 3096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:29.331942 (+ 5896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:29.331951 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:29.331966 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:29.332550 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:29.333179 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:29.333184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:29.333189 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:29.333446 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:29.334530 (+ 1084us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:29.334539 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:29.334921 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 09:17:29.335203 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 09:17:29.335740 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12606,"thread_start_us":12220,"threads_started":1}
I20260811 09:17:29.342943 7097 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41767
I20260811 09:17:29.343271 7097 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:29.344036 7097 heartbeater.cc:511] Master 127.17.16.126:41767 requested a full tablet report, sending...
W20260811 09:17:29.346712 7097 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:41767 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 09:17:29.346920 7097 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:29.347159 7097 heartbeater.cc:511] Master 127.17.16.126:41767 requested a full tablet report, sending...
I20260811 09:17:29.348338 7097 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:29.348599 7097 heartbeater.cc:511] Master 127.17.16.126:41767 requested a full tablet report, sending...
W20260811 09:17:29.349732 7097 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:17:29.716959 7102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:29.717466 7102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:29.717715 7102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:29.729348 7102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:29.729549 7102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:29.729784 7102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:17:29.745543 7102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.16.84:50239/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:42009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=42987
--tserver_master_addrs=127.17.16.126:41767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:29.747507 7102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:29.749241 7102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:29.766160 7133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:29.766508 7135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:29.773808 7102 server_base.cc:1034] running on GCE node
W20260811 09:17:29.773833 7137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:29.774919 7102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:29.776234 7102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:29.777495 7102 hybrid_clock.cc:648] HybridClock initialized: now 1786439849777420 us; error 61 us; skew 500 ppm
I20260811 09:17:29.777891 7102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:29 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:29.782454 7102 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
Aug 11 09:17:29 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:29.784936 7102 webserver.cc:460] Webserver started at http://127.17.16.66:42987/ using document root <none> and password file <none>
I20260811 09:17:29.785883 7102 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:29.786039 7102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:17:29 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:17:29.835624 7102 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.005s sys 0.004s
I20260811 09:17:29.839303 7151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:29.841277 7102 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:17:29.841464 7102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "f58610f4c6a647db8d2c69ae9f40e05b"
format_stamp: "Formatted at 2026-08-11 09:17:26 on dist-test-slave-ncp9"
server_key: "31cf032c1fa32b2ec829eedbf762a0c4"
server_key_iv: "8497d79a4a4f39da636d89642ccd7266"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:29.842890 7102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:29.863818 7102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:29.869087 7102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:29.869472 7102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:29.870787 7102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:29.872613 7102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:29.872754 7102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:29.872875 7102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:29.873004 7102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:29.918397 7102 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:42009
I20260811 09:17:29.918452 7324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:42009 every 8 connection(s)
I20260811 09:17:29.919857 7102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:17:29.929394 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 7102
I20260811 09:17:29.930190 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.16.84:50239/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:32885
--local_ip_for_outbound_sockets=127.17.16.67
--tserver_master_addrs=127.17.16.126:41767
--webserver_port=42531
--webserver_interface=127.17.16.67
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.16.84:50239/kms with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:17:29 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:29.958380 7334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:29.924117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:53751 (local address 127.17.16.126:41767)
0811 09:17:29.933404 (+ 9287us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:29.933412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:29.936583 (+ 3171us) server_negotiation.cc:407] Connection header received
0811 09:17:29.936724 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:29.936730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:29.937137 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:29.937277 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:29.940653 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:29.941859 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:29.944728 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:29.945562 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:29.945667 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:29.951050 (+ 5383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:29.951056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:29.951059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:29.951095 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:29.954421 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:29.955728 (+ 1307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:29.955734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:29.955737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:29.955898 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:29.956543 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:29.956549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:29.956553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:29.956852 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:29.957030 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:29.957794 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 09:17:29.958022 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9100,"thread_start_us":8976,"threads_started":1}
I20260811 09:17:29.959583 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:29.923439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41767 (local address 127.17.16.66:53751)
0811 09:17:29.933999 (+ 10560us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:29.934047 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:29.936183 (+ 2136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:29.938841 (+ 2658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:29.938854 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:29.939505 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:29.940423 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:29.940453 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:29.942088 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:29.942094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:29.944527 (+ 2433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:29.944545 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:29.946089 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:29.946099 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:29.946481 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:29.947377 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:29.947410 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:29.950799 (+ 3389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:29.954608 (+ 3809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:29.954619 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:29.954638 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:29.955575 (+ 937us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:29.956059 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:29.956063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:29.956066 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:29.956345 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:29.957025 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:29.957033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:29.957405 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 09:17:29.957681 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 09:17:29.958294 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10272,"thread_start_us":10008,"threads_started":1}
I20260811 09:17:29.961149 7326 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41767
I20260811 09:17:29.961416 7326 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:29.962368 7326 heartbeater.cc:511] Master 127.17.16.126:41767 requested a full tablet report, sending...
I20260811 09:17:29.964767 6794 ts_manager.cc:194] Registered new tserver with Master: f58610f4c6a647db8d2c69ae9f40e05b (127.17.16.66:42009)
I20260811 09:17:29.966975 6794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:53751
Aug 11 09:17:30 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
W20260811 09:17:30.348909 7335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:30.349233 7335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:30.349391 7335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 09:17:30.349884 7097 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:30.350297 7097 heartbeater.cc:511] Master 127.17.16.126:41767 requested a full tablet report, sending...
I20260811 09:17:30.351553 6794 ts_manager.cc:194] Registered new tserver with Master: d3815f99550d4276896e88056ddf0c06 (127.17.16.65:35127)
I20260811 09:17:30.352399 6794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:39903
W20260811 09:17:30.357743 7335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:30.357851 7335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:30.358002 7335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:17:30.368369 7335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.16.84:50239/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:32885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=42531
--tserver_master_addrs=127.17.16.126:41767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:30.369993 7335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:30.371677 7335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:30.387394 7340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:30.388985 7341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:30.397341 7335 server_base.cc:1034] running on GCE node
W20260811 09:17:30.397882 7343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:30.398797 7335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:30.400195 7335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:30.401530 7335 hybrid_clock.cc:648] HybridClock initialized: now 1786439850401463 us; error 52 us; skew 500 ppm
I20260811 09:17:30.401917 7335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:30 dist-test-slave-ncp9 krb5kdc[5103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439850, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:30.406071 7335 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:17:30.409816 7335 webserver.cc:460] Webserver started at http://127.17.16.67:42531/ using document root <none> and password file <none>
I20260811 09:17:30.410790 7335 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:30.410944 7335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:17:30 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439850, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:17:30.449882 7335 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.004s sys 0.000s
I20260811 09:17:30.453508 7355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:30.455310 7335 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 09:17:30.455492 7335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "c3919a65646d48b9b73d0e7d4c63528b"
format_stamp: "Formatted at 2026-08-11 09:17:27 on dist-test-slave-ncp9"
server_key: "8f8c96e6f5fc7ffc5e8fca2414e0573b"
server_key_iv: "ada43d47fe05eac7c70df109aa20bc89"
server_key_version: "kuduclusterkey@0"
I20260811 09:17:30.456935 7335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:30.482786 7335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:30.487851 7335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:30.488149 7335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:30.489385 7335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:30.491130 7335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:30.491245 7335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:30.491360 7335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:30.491448 7335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:17:30 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439849, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for HTTP/127.17.16.84@KRBTEST.COM
I20260811 09:17:30.582090 7335 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:32885
I20260811 09:17:30.582125 7532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:32885 every 8 connection(s)
I20260811 09:17:30.583616 7335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:17:30.592314 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 7335
Aug 11 09:17:30 dist-test-slave-ncp9 krb5kdc[5103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439850, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:30.622799 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.600627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35588 (local address 127.17.16.126:41767)
0811 09:17:30.605711 (+ 5084us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:30.605716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:30.605733 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:17:30.605782 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:30.605787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:30.605874 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:30.605963 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:30.608005 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.609278 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.611112 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.612430 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.612546 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.615515 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:30.615520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:30.615525 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:30.615565 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:30.619601 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:30.620310 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:30.620316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:30.620319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:30.620512 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:30.620917 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:30.620923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:30.620927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:30.621281 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:30.621528 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:30.621834 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 09:17:30.622069 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":4961,"thread_start_us":4747,"threads_started":1}
I20260811 09:17:30.625313 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.588171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41767 (local address 127.17.16.67:59217)
0811 09:17:30.598304 (+ 10133us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:30.598359 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:30.602439 (+ 4080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:30.603510 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:30.603523 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:30.604060 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:30.604796 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.604830 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.606296 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.606302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.608266 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.608279 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.609381 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.609389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.609671 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.610340 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:30.610362 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:30.613088 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:30.619479 (+ 6391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:30.619490 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:30.619511 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:30.620167 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:30.621168 (+ 1001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:30.621173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:30.621178 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:30.621645 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:30.622440 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:30.622449 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:30.622933 (+ 484us) client_negotiation.cc:769] Sending connection context
0811 09:17:30.623256 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 09:17:30.623846 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9779,"thread_start_us":11179,"threads_started":1}
I20260811 09:17:30.626597 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.588824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:59217 (local address 127.17.16.126:41767)
0811 09:17:30.600811 (+ 11987us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:30.600819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:30.600845 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:17:30.602896 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:30.602902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:30.603008 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:30.603106 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:30.605066 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.606147 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.608431 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.609139 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.609275 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.614261 (+ 4986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:30.614268 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:30.614273 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:30.614311 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:30.619294 (+ 4983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:30.620333 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:30.620338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:30.620342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:30.620517 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:30.621796 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:30.621801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:30.621805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:30.622134 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:30.622348 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:30.625982 (+ 3634us) server_negotiation.cc:299] Negotiation successful
0811 09:17:30.626216 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":11835,"thread_start_us":11614,"threads_started":1}
I20260811 09:17:30.627727 7534 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41767
I20260811 09:17:30.628083 7534 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:30.629000 7534 heartbeater.cc:511] Master 127.17.16.126:41767 requested a full tablet report, sending...
I20260811 09:17:30.630601 6794 ts_manager.cc:194] Registered new tserver with Master: c3919a65646d48b9b73d0e7d4c63528b (127.17.16.67:32885)
I20260811 09:17:30.631711 6794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:59217
I20260811 09:17:30.637989 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:17:30.691831 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.664851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.17.16.126:41767)
0811 09:17:30.665072 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:30.665078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:30.673033 (+ 7955us) server_negotiation.cc:407] Connection header received
0811 09:17:30.673351 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:30.673357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:30.673461 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:30.673558 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:30.675813 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.677056 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.679295 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.679953 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.680088 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.683813 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:30.683821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:30.683827 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:30.683874 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:30.687632 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:30.688563 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:30.688569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:30.688572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:30.688735 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:30.689248 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:30.689255 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:30.689259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:30.689664 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:30.689908 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:30.691192 (+ 1284us) server_negotiation.cc:299] Negotiation successful
0811 09:17:30.691435 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 09:17:30.698285 6794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:17:30.709678 6794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:17:30.786634 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.741288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.17.16.65:35127)
0811 09:17:30.762798 (+ 21510us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:30.762805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:30.769065 (+ 6260us) server_negotiation.cc:407] Connection header received
0811 09:17:30.769234 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:30.769241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:30.769463 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:30.769648 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:30.777706 (+ 8058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.779414 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.782728 (+ 3314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.784360 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.784466 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.784728 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:30.786118 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0811 09:17:30.786310 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21361,"thread_start_us":21506,"threads_started":1}
I20260811 09:17:30.787746 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.741005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35127 (local address 127.0.0.1:48766)
0811 09:17:30.768930 (+ 27925us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:30.768949 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:30.769094 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:30.776988 (+ 7894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:30.776994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:30.777028 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:30.777479 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.777492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.779638 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.779644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.782531 (+ 2887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.782549 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.784977 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.784984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.785197 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.785260 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 09:17:30.787169 (+ 1909us) client_negotiation.cc:240] Negotiation successful
0811 09:17:30.787372 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27781,"mutex_wait_us":4,"thread_start_us":26863,"threads_started":1}
I20260811 09:17:30.788090 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.739223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45400 (local address 127.17.16.66:42009)
0811 09:17:30.759056 (+ 19833us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:30.759066 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:30.767790 (+ 8724us) server_negotiation.cc:407] Connection header received
0811 09:17:30.768171 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:30.768181 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:30.768443 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:30.768729 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:30.777268 (+ 8539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.778942 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.784802 (+ 5860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.786229 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.786348 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.786558 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:30.787587 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0811 09:17:30.787761 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19626,"thread_start_us":19828,"threads_started":1}
I20260811 09:17:30.792718 6990 tablet_service.cc:1467] Processing CreateTablet for tablet 75f8fdb6685a4f0ba01f62a647c7e7c0 (DEFAULT_TABLE table=test-table [id=1792f6ddea964d4582ecb4432900fc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:17:30.793684 7557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.739043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:42009 (local address 127.0.0.1:45400)
0811 09:17:30.767568 (+ 28525us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:30.767616 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:30.767851 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:30.769100 (+ 1249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:30.769108 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:30.769159 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:30.775994 (+ 6835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.776013 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.779166 (+ 3153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.779170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.781794 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.781810 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.786959 (+ 5149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.786966 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.787143 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.787177 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:17:30.787259 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 09:17:30.787393 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28229,"thread_start_us":30132,"threads_started":1}
I20260811 09:17:30.794648 6990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f8fdb6685a4f0ba01f62a647c7e7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:30.797418 7230 tablet_service.cc:1467] Processing CreateTablet for tablet 75f8fdb6685a4f0ba01f62a647c7e7c0 (DEFAULT_TABLE table=test-table [id=1792f6ddea964d4582ecb4432900fc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:17:30.798774 7230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f8fdb6685a4f0ba01f62a647c7e7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:30.801842 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.769657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:32885 (local address 127.0.0.1:46032)
0811 09:17:30.785798 (+ 16141us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:30.785817 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:30.785931 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:30.788046 (+ 2115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:30.788051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:30.788079 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:30.793076 (+ 4997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.793089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.795117 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.795122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.797045 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.797061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.801077 (+ 4016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.801084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.801253 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.801287 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:17:30.801368 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 09:17:30.801517 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16026,"thread_start_us":15883,"threads_started":1}
I20260811 09:17:30.802626 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.769981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.17.16.67:32885)
0811 09:17:30.786700 (+ 16719us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:30.786709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:30.786746 (+ 37us) server_negotiation.cc:407] Connection header received
0811 09:17:30.786848 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:30.786856 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:30.787114 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:30.787308 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:30.793387 (+ 6079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.794950 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.798860 (+ 3910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.800503 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.800619 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.800843 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:30.802097 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0811 09:17:30.802288 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16497,"thread_start_us":16571,"threads_started":1}
I20260811 09:17:30.811986 7425 tablet_service.cc:1467] Processing CreateTablet for tablet 75f8fdb6685a4f0ba01f62a647c7e7c0 (DEFAULT_TABLE table=test-table [id=1792f6ddea964d4582ecb4432900fc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:17:30.813830 7425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f8fdb6685a4f0ba01f62a647c7e7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:30.829891 7573 tablet_bootstrap.cc:492] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: Bootstrap starting.
I20260811 09:17:30.834197 7571 tablet_bootstrap.cc:492] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: Bootstrap starting.
I20260811 09:17:30.836540 7573 tablet_bootstrap.cc:654] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:30.837868 7571 tablet_bootstrap.cc:654] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:30.837962 7573 log.cc:824] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:30.839103 7571 log.cc:824] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:30.839340 7574 tablet_bootstrap.cc:492] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: Bootstrap starting.
I20260811 09:17:30.843180 7574 tablet_bootstrap.cc:654] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:30.844424 7574 log.cc:824] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:30.847791 7573 tablet_bootstrap.cc:492] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: No bootstrap required, opened a new log
I20260811 09:17:30.848135 7573 ts_tablet_manager.cc:1397] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.004s
I20260811 09:17:30.851100 7571 tablet_bootstrap.cc:492] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: No bootstrap required, opened a new log
I20260811 09:17:30.851358 7571 ts_tablet_manager.cc:1397] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: Time spent bootstrapping tablet: real 0.017s user 0.017s sys 0.000s
I20260811 09:17:30.858687 7573 raft_consensus.cc:348] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 } } peers { permanent_uuid: "d3815f99550d4276896e88056ddf0c06" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35127 } } peers { permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 } }
I20260811 09:17:30.859117 7573 raft_consensus.cc:374] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:30.859211 7573 raft_consensus.cc:729] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58610f4c6a647db8d2c69ae9f40e05b, State: Initialized, Role: FOLLOWER
I20260811 09:17:30.859918 7573 consensus_queue.cc:260] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 } } peers { permanent_uuid: "d3815f99550d4276896e88056ddf0c06" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35127 } } peers { permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 } }
I20260811 09:17:30.861548 7574 tablet_bootstrap.cc:492] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: No bootstrap required, opened a new log
I20260811 09:17:30.861933 7574 ts_tablet_manager.cc:1397] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.010s
I20260811 09:17:30.861693 7571 raft_consensus.cc:348] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 } } peers { permanent_uuid: "d3815f99550d4276896e88056ddf0c06" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35127 } } peers { permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 } }
I20260811 09:17:30.862159 7571 raft_consensus.cc:374] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:30.862278 7571 raft_consensus.cc:729] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3815f99550d4276896e88056ddf0c06, State: Initialized, Role: FOLLOWER
I20260811 09:17:30.863013 7571 consensus_queue.cc:260] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 } } peers { permanent_uuid: "d3815f99550d4276896e88056ddf0c06" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35127 } } peers { permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 } }
I20260811 09:17:30.864965 7326 heartbeater.cc:503] Master 127.17.16.126:41767 was elected leader, sending a full tablet report...
I20260811 09:17:30.865942 7573 ts_tablet_manager.cc:1428] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
I20260811 09:17:30.866811 7097 heartbeater.cc:503] Master 127.17.16.126:41767 was elected leader, sending a full tablet report...
I20260811 09:17:30.867388 7571 ts_tablet_manager.cc:1428] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: Time spent starting tablet: real 0.016s user 0.011s sys 0.003s
I20260811 09:17:30.873090 7574 raft_consensus.cc:348] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 } } peers { permanent_uuid: "d3815f99550d4276896e88056ddf0c06" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35127 } } peers { permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 } }
I20260811 09:17:30.873394 7574 raft_consensus.cc:374] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:30.873483 7574 raft_consensus.cc:729] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3919a65646d48b9b73d0e7d4c63528b, State: Initialized, Role: FOLLOWER
I20260811 09:17:30.874004 7574 consensus_queue.cc:260] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 } } peers { permanent_uuid: "d3815f99550d4276896e88056ddf0c06" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35127 } } peers { permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 } }
I20260811 09:17:30.874987 7534 heartbeater.cc:503] Master 127.17.16.126:41767 was elected leader, sending a full tablet report...
I20260811 09:17:30.875664 7574 ts_tablet_manager.cc:1428] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
I20260811 09:17:30.888054 7593 raft_consensus.cc:482] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:17:30.888290 7593 raft_consensus.cc:504] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 } } peers { permanent_uuid: "d3815f99550d4276896e88056ddf0c06" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35127 } } peers { permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 } }
I20260811 09:17:30.889822 7593 leader_election.cc:288] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f58610f4c6a647db8d2c69ae9f40e05b (127.17.16.66:42009), c3919a65646d48b9b73d0e7d4c63528b (127.17.16.67:32885)
I20260811 09:17:30.896201 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.890252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:58611 (local address 127.17.16.66:42009)
0811 09:17:30.890478 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:30.890485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:30.890832 (+ 347us) server_negotiation.cc:407] Connection header received
0811 09:17:30.891054 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:30.891061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:30.891170 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:30.891254 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:30.891913 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.892730 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.894545 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.895343 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.895486 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.895559 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:30.895838 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 09:17:30.895959 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:17:30.896265 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.891075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:44863 (local address 127.17.16.67:32885)
0811 09:17:30.891261 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:30.891266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:30.891424 (+ 158us) server_negotiation.cc:407] Connection header received
0811 09:17:30.891585 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:30.891588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:30.891642 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:30.891713 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:30.892218 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.893047 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.894708 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.895441 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.895590 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.895658 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:30.895928 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 09:17:30.896044 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 09:17:30.896397 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.890834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:32885 (local address 127.17.16.65:44863)
0811 09:17:30.891332 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:30.891345 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:30.891456 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:30.891750 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:30.891755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:30.891781 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:30.892085 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.892093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.893172 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.893176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.894570 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.894582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.895654 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.895661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.895791 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.895813 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 09:17:30.895889 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 09:17:30.896006 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":1,"thread_start_us":429,"threads_started":1}
I20260811 09:17:30.896651 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.890142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:42009 (local address 127.17.16.65:58611)
0811 09:17:30.890725 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:30.890740 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:30.890882 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:30.891415 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:30.891420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:30.891491 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:30.891774 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.891783 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.892858 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.892862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.894400 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:30.894412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.895541 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.895547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:30.895694 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.895717 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:17:30.895801 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 09:17:30.895919 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":57,"spinlock_wait_cycles":15232,"thread_start_us":310,"threads_started":1}
I20260811 09:17:30.897400 7461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f8fdb6685a4f0ba01f62a647c7e7c0" candidate_uuid: "d3815f99550d4276896e88056ddf0c06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3919a65646d48b9b73d0e7d4c63528b" is_pre_election: true
I20260811 09:17:30.897625 7253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f8fdb6685a4f0ba01f62a647c7e7c0" candidate_uuid: "d3815f99550d4276896e88056ddf0c06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" is_pre_election: true
I20260811 09:17:30.897868 7461 raft_consensus.cc:2454] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3815f99550d4276896e88056ddf0c06 in term 0.
I20260811 09:17:30.898092 7253 raft_consensus.cc:2454] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3815f99550d4276896e88056ddf0c06 in term 0.
I20260811 09:17:30.898555 6919 leader_election.cc:302] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3919a65646d48b9b73d0e7d4c63528b, d3815f99550d4276896e88056ddf0c06; no voters:
I20260811 09:17:30.899544 7593 raft_consensus.cc:2781] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:17:30.899657 7593 raft_consensus.cc:482] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:17:30.899730 7593 raft_consensus.cc:3037] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:30.902009 7593 raft_consensus.cc:504] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 } } peers { permanent_uuid: "d3815f99550d4276896e88056ddf0c06" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35127 } } peers { permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 } }
I20260811 09:17:30.902649 7593 leader_election.cc:288] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [CANDIDATE]: Term 1 election: Requested vote from peers f58610f4c6a647db8d2c69ae9f40e05b (127.17.16.66:42009), c3919a65646d48b9b73d0e7d4c63528b (127.17.16.67:32885)
I20260811 09:17:30.903007 7253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f8fdb6685a4f0ba01f62a647c7e7c0" candidate_uuid: "d3815f99550d4276896e88056ddf0c06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f58610f4c6a647db8d2c69ae9f40e05b"
I20260811 09:17:30.903093 7461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f8fdb6685a4f0ba01f62a647c7e7c0" candidate_uuid: "d3815f99550d4276896e88056ddf0c06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3919a65646d48b9b73d0e7d4c63528b"
I20260811 09:17:30.903214 7253 raft_consensus.cc:3037] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:30.903290 7461 raft_consensus.cc:3037] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:30.905445 7253 raft_consensus.cc:2454] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3815f99550d4276896e88056ddf0c06 in term 1.
I20260811 09:17:30.905499 7461 raft_consensus.cc:2454] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3815f99550d4276896e88056ddf0c06 in term 1.
I20260811 09:17:30.905777 6919 leader_election.cc:302] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3815f99550d4276896e88056ddf0c06, f58610f4c6a647db8d2c69ae9f40e05b; no voters:
I20260811 09:17:30.906029 7593 raft_consensus.cc:2781] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:17:30.906445 7593 raft_consensus.cc:686] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 1 LEADER]: Becoming Leader. State: Replica: d3815f99550d4276896e88056ddf0c06, State: Running, Role: LEADER
I20260811 09:17:30.906872 7593 consensus_queue.cc:237] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 } } peers { permanent_uuid: "d3815f99550d4276896e88056ddf0c06" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35127 } } peers { permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 } }
I20260811 09:17:30.911238 6794 catalog_manager.cc:5495] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3815f99550d4276896e88056ddf0c06 (127.17.16.65). New cstate: current_term: 1 leader_uuid: "d3815f99550d4276896e88056ddf0c06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3815f99550d4276896e88056ddf0c06" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 } health_report { overall_health: UNKNOWN } } }
W20260811 09:17:30.928406 7330 tablet.cc:2369] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:17:30.940840 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.935849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.17.16.65:35127)
0811 09:17:30.936025 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:30.936029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:30.936047 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:17:30.936210 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:30.936213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:30.936296 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:30.936409 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:17:30.936699 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.937473 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.939090 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.939470 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.939561 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:30.939972 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:17:30.940418 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:17:30.940525 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:30.940591 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 09:17:30.940651 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:17:30.948240 7253 raft_consensus.cc:1261] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b [term 1 FOLLOWER]: Refusing update from remote peer d3815f99550d4276896e88056ddf0c06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:17:30.948371 7461 raft_consensus.cc:1261] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b [term 1 FOLLOWER]: Refusing update from remote peer d3815f99550d4276896e88056ddf0c06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:17:30.949251 7600 consensus_queue.cc:1035] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f58610f4c6a647db8d2c69ae9f40e05b" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:17:30.949672 7593 consensus_queue.cc:1035] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3919a65646d48b9b73d0e7d4c63528b" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:17:30.962982 7608 mvcc.cc:205] Tried to move back new op lower bound from 7317257629475266560 to 7317257629317849088. Current Snapshot: MvccSnapshot[applied={T|T < 7317257629475266560 or (T in {7317257629475266560})}]
I20260811 09:17:30.974160 6794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:17:30.974429 6794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:17:30.977764 6794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0ce0d96016645f491207306d7f93eaf: Sending DeleteTablet for 3 replicas of tablet 75f8fdb6685a4f0ba01f62a647c7e7c0
I20260811 09:17:30.978770 6990 tablet_service.cc:1514] Processing DeleteTablet for tablet 75f8fdb6685a4f0ba01f62a647c7e7c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:17:30 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:48766
I20260811 09:17:30.979264 7425 tablet_service.cc:1514] Processing DeleteTablet for tablet 75f8fdb6685a4f0ba01f62a647c7e7c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:17:30 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:46032
I20260811 09:17:30.979455 7230 tablet_service.cc:1514] Processing DeleteTablet for tablet 75f8fdb6685a4f0ba01f62a647c7e7c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:17:30 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:45400
I20260811 09:17:30.979708 7623 tablet_replica.cc:330] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: stopping tablet replica
I20260811 09:17:30.980156 7623 raft_consensus.cc:2229] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:17:30.980819 7623 raft_consensus.cc:2258] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:17:30.983630 7623 ts_tablet_manager.cc:1905] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:17:30.989378 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 6848
I20260811 09:17:30.992192 7623 ts_tablet_manager.cc:1918] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:17:30.992383 7623 log.cc:1196] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/75f8fdb6685a4f0ba01f62a647c7e7c0
I20260811 09:17:30.993151 7623 ts_tablet_manager.cc:1939] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P d3815f99550d4276896e88056ddf0c06: Deleting consensus metadata
I20260811 09:17:30.996666 6769 catalog_manager.cc:4841] TS d3815f99550d4276896e88056ddf0c06 (127.17.16.65:35127): tablet 75f8fdb6685a4f0ba01f62a647c7e7c0 (table test-table [id=1792f6ddea964d4582ecb4432900fc48]) successfully deleted
I20260811 09:17:31.006384 7627 tablet_replica.cc:330] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: stopping tablet replica
I20260811 09:17:31.007380 7627 raft_consensus.cc:2229] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:17:31.007444 7625 tablet_replica.cc:330] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: stopping tablet replica
I20260811 09:17:31.007951 7627 raft_consensus.cc:2258] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:17:31.008977 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:30.990772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48786 (local address 127.17.16.65:35127)
0811 09:17:30.990957 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:30.990964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:30.990989 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:17:30.991188 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:30.991194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:30.991285 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:30.991387 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:30.993570 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.994821 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.997066 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:30.997748 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:30.997923 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:31.001539 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:31.001554 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:31.001578 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:31.001627 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:31.005668 (+ 4041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:31.006526 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:31.006536 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:31.006539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:31.006738 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:31.007185 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:31.007191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:31.007195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:31.007673 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:31.008028 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:31.008312 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 09:17:31.008582 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:17:31.008968 7625 raft_consensus.cc:2229] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:17:31.009446 7625 raft_consensus.cc:2258] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:17:31.009907 7092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:31.011336 7627 ts_tablet_manager.cc:1905] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:17:31.011523 7625 ts_tablet_manager.cc:1905] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:17:31.017501 7625 ts_tablet_manager.cc:1918] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:17:31.017647 7625 log.cc:1196] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/75f8fdb6685a4f0ba01f62a647c7e7c0
I20260811 09:17:31.018055 7625 ts_tablet_manager.cc:1939] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P c3919a65646d48b9b73d0e7d4c63528b: Deleting consensus metadata
I20260811 09:17:31.019536 6771 catalog_manager.cc:4841] TS c3919a65646d48b9b73d0e7d4c63528b (127.17.16.67:32885): tablet 75f8fdb6685a4f0ba01f62a647c7e7c0 (table test-table [id=1792f6ddea964d4582ecb4432900fc48]) successfully deleted
I20260811 09:17:31.020097 7627 ts_tablet_manager.cc:1918] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:17:31.020284 7627 log.cc:1196] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/75f8fdb6685a4f0ba01f62a647c7e7c0
I20260811 09:17:31.020810 7627 ts_tablet_manager.cc:1939] T 75f8fdb6685a4f0ba01f62a647c7e7c0 P f58610f4c6a647db8d2c69ae9f40e05b: Deleting consensus metadata
I20260811 09:17:31.022629 6771 catalog_manager.cc:4841] TS f58610f4c6a647db8d2c69ae9f40e05b (127.17.16.66:42009): tablet 75f8fdb6685a4f0ba01f62a647c7e7c0 (table test-table [id=1792f6ddea964d4582ecb4432900fc48]) successfully deleted
I20260811 09:17:31.108105 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 6848
I20260811 09:17:31.130604 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 7102
I20260811 09:17:31.142870 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:31.131313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45414 (local address 127.17.16.66:42009)
0811 09:17:31.131466 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:31.131470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:31.131492 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:17:31.131660 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:31.131664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:31.131725 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:31.131805 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:31.133209 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:31.133933 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:31.135103 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:31.135475 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:31.135560 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:31.137932 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:31.137944 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:31.137959 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:31.137992 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:31.140591 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:31.141218 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:31.141225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:31.141227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:31.141346 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:31.141626 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:31.141629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:31.141631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:31.141987 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:31.142224 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:31.142439 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 09:17:31.142621 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 09:17:31.143494 7317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:31.230369 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 7102
I20260811 09:17:31.251510 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 7335
I20260811 09:17:31.264097 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:31.252250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.17.16.67:32885)
0811 09:17:31.252454 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:31.252459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:31.252477 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:17:31.252591 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:31.252595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:31.252661 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:31.252739 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:31.254010 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:31.254752 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:31.255954 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:31.256332 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:31.256414 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:31.258777 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:31.258793 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:31.258813 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:31.258847 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:31.261395 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:31.262014 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:31.262028 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:31.262031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:31.262163 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:31.262546 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:31.262551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:31.262554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:31.262957 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:31.263267 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:31.263536 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 09:17:31.263757 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 09:17:31.264745 7527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:31.351258 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 7335
I20260811 09:17:31.375519 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 6745
I20260811 09:17:31.376731 6838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:31.487861 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 6745
I20260811 09:17:31.521565 17473 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 09:17:31.865653 17473 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 09:17:31.865867 17473 mini_ranger.cc:66] Stopping Ranger...
I20260811 09:17:32.209816 17473 mini_ranger.cc:68] Stopped Ranger
2026-08-11 09:17:32.210 UTC [5149] LOG: received smart shutdown request
2026-08-11 09:17:32.213 UTC [5149] LOG: background worker "logical replication launcher" (PID 5166) exited with exit code 1
2026-08-11 09:17:32.213 UTC [5157] LOG: shutting down
2026-08-11 09:17:32.348 UTC [5149] LOG: database system is shut down
2026-08-11T09:17:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50724 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:17:32 dist-test-slave-ncp9 krb5kdc[7645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:17:32 dist-test-slave-ncp9 krb5kdc[7645](info): set up 2 sockets
Aug 11 09:17:32 dist-test-slave-ncp9 krb5kdc[7645](info): commencing operation
krb5kdc: starting...
W20260811 09:17:33.686467 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:17:33 dist-test-slave-ncp9 krb5kdc[7645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:17:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:17:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:17:33.976768 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46851
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:44159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:46851,127.17.16.125:36951,127.17.16.124:40363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:17:34.306231 7671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:34.306499 7671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:34.306628 7671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:34.313539 7671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:17:34.313637 7671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:34.313711 7671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:34.313781 7671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:17:34.313838 7671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:17:34.323334 7671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:44159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:46851,127.17.16.125:36951,127.17.16.124:40363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:46851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:34.324594 7671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:34.325883 7671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:34.336738 7677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:34.336767 7679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:34.336792 7683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:34.337961 7671 server_base.cc:1034] running on GCE node
I20260811 09:17:34.338656 7671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:34.339905 7671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:34.341091 7671 hybrid_clock.cc:648] HybridClock initialized: now 1786439854341049 us; error 27 us; skew 500 ppm
I20260811 09:17:34.341406 7671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:34 dist-test-slave-ncp9 krb5kdc[7645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:34.348296 7671 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:17:34.349730 7671 webserver.cc:460] Webserver started at http://127.17.16.126:38893/ using document root <none> and password file <none>
I20260811 09:17:34.350389 7671 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:34.350504 7671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:34.350795 7671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:17:34.353770 7671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "150b8d13ac5e45aa8b6b633fa0c74858"
format_stamp: "Formatted at 2026-08-11 09:17:34 on dist-test-slave-ncp9"
server_key: "d3aa8b423d2efea50636d57a56c21a0c"
server_key_iv: "af86d939c67f1ecdc8441776a38ba12d"
server_key_version: "encryptionkey@0"
I20260811 09:17:34.354553 7671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "150b8d13ac5e45aa8b6b633fa0c74858"
format_stamp: "Formatted at 2026-08-11 09:17:34 on dist-test-slave-ncp9"
server_key: "d3aa8b423d2efea50636d57a56c21a0c"
server_key_iv: "af86d939c67f1ecdc8441776a38ba12d"
server_key_version: "encryptionkey@0"
I20260811 09:17:34.359561 7671 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:17:34.362942 7692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:34.364197 7671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 09:17:34.364357 7671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "150b8d13ac5e45aa8b6b633fa0c74858"
format_stamp: "Formatted at 2026-08-11 09:17:34 on dist-test-slave-ncp9"
server_key: "d3aa8b423d2efea50636d57a56c21a0c"
server_key_iv: "af86d939c67f1ecdc8441776a38ba12d"
server_key_version: "encryptionkey@0"
I20260811 09:17:34.364534 7671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:34.376700 7671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:34.380401 7671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:34.380657 7671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:34.397714 7671 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:46851
I20260811 09:17:34.397733 7763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:46851 every 8 connection(s)
I20260811 09:17:34.399034 7671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:17:34.401391 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 7671
I20260811 09:17:34.401636 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:17:34.402494 17473 external_mini_cluster.cc:1428] Setting key f980a1681704d48f2c1cff507ce83026
I20260811 09:17:34.403272 7764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:34.411687 7764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:17:34 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.16.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.125@KRBTEST.COM" created.
I20260811 09:17:34.434873 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.415099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59556 (local address 127.17.16.126:46851)
0811 09:17:34.416305 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:34.416311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:34.416350 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:17:34.418647 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:34.418653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:34.418867 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:34.419019 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:34.420514 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.421770 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.423634 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.424157 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.424287 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:34.428104 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:34.428112 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:34.428123 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:34.428151 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:34.430708 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:34.431551 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:34.431558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:34.431560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:34.431699 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:34.432092 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:34.432097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:34.432099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:34.432724 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:34.432946 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:34.433499 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 09:17:34.433844 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":1023,"spinlock_wait_cycles":16128,"thread_start_us":845,"threads_started":1}
I20260811 09:17:34.434909 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.415098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46851 (local address 127.0.0.1:59556)
0811 09:17:34.415905 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.415963 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:34.418276 (+ 2313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:34.419118 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:34.419124 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:34.419833 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:34.420312 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:34.420336 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.421935 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.421940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:34.423507 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:34.423521 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.424401 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.424408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:34.424565 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:34.425227 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:34.425253 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:34.427886 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:34.430841 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:34.430854 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:34.430866 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:34.431407 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:34.431804 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:34.431808 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:34.431810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:34.431962 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:34.432868 (+ 906us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:34.432874 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:34.433082 (+ 208us) client_negotiation.cc:769] Sending connection context
0811 09:17:34.433295 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 09:17:34.433685 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":348,"threads_started":1}
I20260811 09:17:34.437963 7764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:17:34.439036 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.438589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56496)
0811 09:17:34.438783 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.438909 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 09:17:34.439293 7699 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
I20260811 09:17:34.440912 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.440512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56504)
0811 09:17:34.440667 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.440744 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 09:17:34.441094 7764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:36951: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab.
Entry for principal kudu/127.17.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab.
I20260811 09:17:34.464604 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.125:36951
--webserver_interface=127.17.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab
--principal=kudu/127.17.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:44159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:46851,127.17.16.125:36951,127.17.16.124:40363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:17:34.490500 7764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } attempt: 1
I20260811 09:17:34.491629 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.491203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56506)
0811 09:17:34.491389 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.491495 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 09:17:34.492586 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.492243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56508)
0811 09:17:34.492387 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.492455 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 09:17:34.492771 7764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:36951: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
I20260811 09:17:34.561012 7764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } attempt: 2
I20260811 09:17:34.562140 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.561663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56520)
0811 09:17:34.561904 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.561996 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 09:17:34.563208 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.562843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56524)
0811 09:17:34.562996 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.563075 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 09:17:34.563375 7764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:36951: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
I20260811 09:17:34.654594 7764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } attempt: 3
I20260811 09:17:34.655792 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.655356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56536)
0811 09:17:34.655554 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.655657 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 09:17:34.656813 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.656445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56544)
0811 09:17:34.656589 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.656681 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 09:17:34.657011 7764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:36951: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
W20260811 09:17:34.784986 7774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:34.785270 7774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:34.785404 7774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:34.792302 7774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:17:34.792410 7774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:34.792495 7774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:34.792584 7774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:17:34.792644 7774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:17:34.800230 7764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } attempt: 4
I20260811 09:17:34.801352 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.800819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56548)
0811 09:17:34.801106 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.801216 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260811 09:17:34.802353 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.801987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56554)
0811 09:17:34.802135 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.802239 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 09:17:34.802536 7764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:36951: Network error: Client connection negotiation failed: client connection to 127.17.16.125:36951: connect: Connection refused (error 111)
I20260811 09:17:34.802498 7774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:44159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:46851,127.17.16.125:36951,127.17.16.124:40363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab
--principal=kudu/127.17.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.125:36951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:34.803730 7774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:34.805042 7774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:34.815980 7782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:34.815899 7785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:34.815953 7783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:34.817093 7774 server_base.cc:1034] running on GCE node
I20260811 09:17:34.817813 7774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:34.819063 7774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:34.820318 7774 hybrid_clock.cc:648] HybridClock initialized: now 1786439854820241 us; error 37 us; skew 500 ppm
I20260811 09:17:34.820734 7774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:34 dist-test-slave-ncp9 krb5kdc[7645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:34.823881 7774 init.cc:377] Logged in from keytab as kudu/127.17.16.125@KRBTEST.COM (short username kudu)
I20260811 09:17:34.825363 7774 webserver.cc:460] Webserver started at http://127.17.16.125:35791/ using document root <none> and password file <none>
I20260811 09:17:34.826087 7774 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:34.826224 7774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:34.826565 7774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:17:34.829697 7774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/data/instance:
uuid: "a2b75da6945d4863b244cdf12b4237cd"
format_stamp: "Formatted at 2026-08-11 09:17:34 on dist-test-slave-ncp9"
server_key: "7c928cf36bed7a1ec7c066c24c7d205f"
server_key_iv: "a184e726e45c86f2f0c3febf461ec90d"
server_key_version: "encryptionkey@0"
I20260811 09:17:34.830550 7774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/wal/instance:
uuid: "a2b75da6945d4863b244cdf12b4237cd"
format_stamp: "Formatted at 2026-08-11 09:17:34 on dist-test-slave-ncp9"
server_key: "7c928cf36bed7a1ec7c066c24c7d205f"
server_key_iv: "a184e726e45c86f2f0c3febf461ec90d"
server_key_version: "encryptionkey@0"
I20260811 09:17:34.835779 7774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:17:34.839260 7795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:34.840467 7774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:17:34.840641 7774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/wal
uuid: "a2b75da6945d4863b244cdf12b4237cd"
format_stamp: "Formatted at 2026-08-11 09:17:34 on dist-test-slave-ncp9"
server_key: "7c928cf36bed7a1ec7c066c24c7d205f"
server_key_iv: "a184e726e45c86f2f0c3febf461ec90d"
server_key_version: "encryptionkey@0"
I20260811 09:17:34.840824 7774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:34.859195 7774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:34.862849 7774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:34.863092 7774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:34.879647 7774 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.125:36951
I20260811 09:17:34.879675 7873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.125:36951 every 8 connection(s)
I20260811 09:17:34.880932 7774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
I20260811 09:17:34.884251 7874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:34.886947 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 7774
I20260811 09:17:34.887137 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-1/wal/instance
I20260811 09:17:34.887694 17473 external_mini_cluster.cc:1428] Setting key 56b8a6d941c75034edea4ce866570a75
I20260811 09:17:34.892594 7874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:17:34 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:34.914242 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.895827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.17.16.126:46851)
0811 09:17:34.896018 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:34.896026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:34.897618 (+ 1592us) server_negotiation.cc:407] Connection header received
0811 09:17:34.899458 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:34.899464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:34.899553 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:34.899681 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:34.901230 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.902027 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.904182 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.904685 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.904795 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:34.908647 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:34.908653 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:34.908656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:34.908691 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:34.911181 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:34.912051 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:34.912056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:34.912058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:34.912177 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:34.912609 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:34.912612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:34.912614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:34.912871 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:34.913011 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:34.913841 (+ 830us) server_negotiation.cc:299] Negotiation successful
0811 09:17:34.914005 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 09:17:34.915584 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.896337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46851 (local address 127.0.0.1:59570)
0811 09:17:34.897442 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.897494 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:34.899192 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:34.899848 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:34.899855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:34.900460 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:34.901044 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:34.901064 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.902218 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.902223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:34.904026 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:34.904040 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.904949 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.904957 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:34.905200 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:34.905861 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:34.905889 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:34.908417 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:34.911333 (+ 2916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:34.911342 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:34.911357 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:34.911937 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:34.912275 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:34.912280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:34.912284 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:34.912515 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:34.913028 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:34.913036 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:34.913454 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 09:17:34.913766 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 09:17:34.914248 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"thread_start_us":620,"threads_started":1}
WARNING: no policy specified for kudu/127.17.16.124@KRBTEST.COM; defaulting to no policy
I20260811 09:17:34.917313 7874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.124@KRBTEST.COM" created.
Aug 11 09:17:34 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:17:34.930550 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.917975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56564)
0811 09:17:34.918158 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.918173 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:34.918267 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:34.919208 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:34.919213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:34.919614 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:34.919745 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:34.919753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.921203 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.921208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:34.922026 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:34.922037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.922824 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.922829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:34.923008 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:34.923507 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:34.923523 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:34.925508 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:34.928358 (+ 2850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:34.928362 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:34.928365 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:34.928713 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:34.929122 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:34.929125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:34.929128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:34.929246 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:34.929902 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:34.929906 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:34.930012 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 09:17:34.930121 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 09:17:34.930301 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 09:17:34.930922 7879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.918181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56564 (local address 127.17.16.125:36951)
0811 09:17:34.918791 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:34.918799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:34.918828 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:17:34.918874 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:34.918879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:34.919072 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:34.919207 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:34.919893 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.921078 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.922159 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:34.922690 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:34.922802 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:34.925723 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:34.925734 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:34.925747 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:34.925780 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:34.928217 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:34.928833 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:34.928842 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:34.928845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:34.929013 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:34.929346 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:34.929351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:34.929354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:34.929765 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:34.929963 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:34.930288 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 09:17:34.930531 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":338,"threads_started":1}
I20260811 09:17:34.933327 7874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:17:34.934343 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.933936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45224)
0811 09:17:34.934091 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.934207 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 09:17:34.934552 7796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
I20260811 09:17:34.935985 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.935669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45238)
0811 09:17:34.935822 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.935891 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 09:17:34.936173 7874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40363: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab.
Entry for principal kudu/127.17.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab.
I20260811 09:17:34.950002 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.124:40363
--webserver_interface=127.17.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab
--principal=kudu/127.17.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:44159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:46851,127.17.16.125:36951,127.17.16.124:40363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:17:34.985440 7874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } attempt: 1
I20260811 09:17:34.986567 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.986126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45240)
0811 09:17:34.986314 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.986417 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 09:17:34.987514 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:34.987178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45244)
0811 09:17:34.987326 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:34.987409 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 09:17:34.987701 7874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40363: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
I20260811 09:17:35.055886 7874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } attempt: 2
I20260811 09:17:35.057026 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.056551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45250)
0811 09:17:35.056739 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.056831 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 09:17:35.058008 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.057671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45252)
0811 09:17:35.057823 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.057895 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:17:35.058211 7874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40363: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
I20260811 09:17:35.101766 7764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } attempt: 5
Aug 11 09:17:35 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:17:35.114804 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.102557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56576)
0811 09:17:35.102781 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.102803 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:35.102944 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:35.103349 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:35.103354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:35.103758 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:35.103952 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.103962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.105007 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.105013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.105887 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.105901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.106693 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.106699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.106869 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.107489 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:35.107507 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:35.109542 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:35.112444 (+ 2902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.112450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.112454 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.112819 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.113194 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.113199 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.113203 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.113338 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.113854 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:35.113859 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:35.114004 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 09:17:35.114131 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 09:17:35.114446 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 09:17:35.114816 7879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.102584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56576 (local address 127.17.16.125:36951)
0811 09:17:35.102782 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.102788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.102877 (+ 89us) server_negotiation.cc:407] Connection header received
0811 09:17:35.103071 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.103075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.103185 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.103273 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.104131 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.104838 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.106043 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.106533 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.106647 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.109760 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.109765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.109768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.109803 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.112311 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.112957 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.112961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.112964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.113088 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.113444 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.113448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.113450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.113738 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.113871 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.114259 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.114486 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:17:35.116053 7764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:17:35.117182 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.116789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45264)
0811 09:17:35.116971 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.117046 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 09:17:35.118055 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.117786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45276)
0811 09:17:35.117891 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.117952 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 09:17:35.118239 7764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40363: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
I20260811 09:17:35.149396 7874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } attempt: 3
I20260811 09:17:35.150395 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.149949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45288)
0811 09:17:35.150155 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.150245 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:17:35.151480 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.151073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45302)
0811 09:17:35.151288 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.151364 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260811 09:17:35.151686 7874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40363: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
I20260811 09:17:35.169430 7764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } attempt: 1
I20260811 09:17:35.170487 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.170073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45316)
0811 09:17:35.170242 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.170324 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 09:17:35.171391 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.171108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45328)
0811 09:17:35.171220 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.171298 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 09:17:35.171583 7764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40363: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
I20260811 09:17:35.239774 7764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } attempt: 2
I20260811 09:17:35.240974 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.240462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45332)
0811 09:17:35.240685 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.240783 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 09:17:35.241894 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.241591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45342)
0811 09:17:35.241715 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.241782 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 09:17:35.242105 7764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40363: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
W20260811 09:17:35.277652 7882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:35.277899 7882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:35.278025 7882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:35.284957 7882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:17:35.285055 7882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:35.285130 7882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:35.285198 7882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:17:35.285252 7882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:17:35.294638 7882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:44159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:46851,127.17.16.125:36951,127.17.16.124:40363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab
--principal=kudu/127.17.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.124:40363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:35.294886 7874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } attempt: 4
I20260811 09:17:35.296005 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.295538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45350)
0811 09:17:35.295766 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.295851 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 09:17:35.296025 7882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:35.297010 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.296636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45360)
0811 09:17:35.296788 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.296861 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 09:17:35.297205 7874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40363: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
I20260811 09:17:35.297505 7882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:35.308547 7891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:35.308483 7895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:35.308493 7893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:35.309391 7882 server_base.cc:1034] running on GCE node
I20260811 09:17:35.310101 7882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:35.311342 7882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:35.312528 7882 hybrid_clock.cc:648] HybridClock initialized: now 1786439855312475 us; error 48 us; skew 500 ppm
I20260811 09:17:35.312839 7882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:35 dist-test-slave-ncp9 krb5kdc[7645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439855, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:35.315804 7882 init.cc:377] Logged in from keytab as kudu/127.17.16.124@KRBTEST.COM (short username kudu)
I20260811 09:17:35.317149 7882 webserver.cc:460] Webserver started at http://127.17.16.124:32937/ using document root <none> and password file <none>
I20260811 09:17:35.317776 7882 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:35.317890 7882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:35.318176 7882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:17:35.320976 7882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/data/instance:
uuid: "1c4c90f403a543ecbb64e86764d9e4c4"
format_stamp: "Formatted at 2026-08-11 09:17:35 on dist-test-slave-ncp9"
server_key: "9fdad50716463b5d5a1c95f299afc467"
server_key_iv: "694b3108a3dd4f166e81edf58c0e41c3"
server_key_version: "encryptionkey@0"
I20260811 09:17:35.321754 7882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/wal/instance:
uuid: "1c4c90f403a543ecbb64e86764d9e4c4"
format_stamp: "Formatted at 2026-08-11 09:17:35 on dist-test-slave-ncp9"
server_key: "9fdad50716463b5d5a1c95f299afc467"
server_key_iv: "694b3108a3dd4f166e81edf58c0e41c3"
server_key_version: "encryptionkey@0"
I20260811 09:17:35.326481 7882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:17:35.329898 7909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:35.331103 7882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:17:35.331254 7882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/wal
uuid: "1c4c90f403a543ecbb64e86764d9e4c4"
format_stamp: "Formatted at 2026-08-11 09:17:35 on dist-test-slave-ncp9"
server_key: "9fdad50716463b5d5a1c95f299afc467"
server_key_iv: "694b3108a3dd4f166e81edf58c0e41c3"
server_key_version: "encryptionkey@0"
I20260811 09:17:35.331409 7882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:35.348287 7764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } attempt: 3
I20260811 09:17:35.349275 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.348859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45374)
0811 09:17:35.349039 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.349123 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 09:17:35.350170 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.349830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45390)
0811 09:17:35.349999 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.350069 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 09:17:35.350342 7764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40363: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40363: connect: Connection refused (error 111)
I20260811 09:17:35.363404 7882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:35.367117 7882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:35.367362 7882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:35.384509 7882 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.124:40363
I20260811 09:17:35.384545 7989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.124:40363 every 8 connection(s)
I20260811 09:17:35.385782 7882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
I20260811 09:17:35.387658 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 7882
I20260811 09:17:35.387872 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439649204149-17473-0/minicluster-data/master-2/wal/instance
I20260811 09:17:35.388491 17473 external_mini_cluster.cc:1428] Setting key b5f0ff2d3c6c11777036bfd8b385ee4d
I20260811 09:17:35.389957 7990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:35.398609 7990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:17:35 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
Aug 11 09:17:35 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439855, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:35.412914 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.390339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45398 (local address 127.17.16.124:40363)
0811 09:17:35.391076 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.391086 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.391127 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:17:35.392644 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.392665 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.393081 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.393403 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.395161 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.396507 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.398422 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.399432 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.399592 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.404835 (+ 5243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.404843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.404853 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.404918 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.408096 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.408949 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.408955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.408960 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.409118 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.409500 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.409504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.409507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.410155 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.410364 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.410847 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.411616 (+ 769us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":352,"threads_started":1}
I20260811 09:17:35.417219 7996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.401348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:46851 (local address 127.0.0.1:59574)
0811 09:17:35.402133 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.402201 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:35.402407 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:35.402968 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:35.402972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:35.403481 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:35.403788 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.403798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.404825 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.404831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.406539 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.406554 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.407535 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.407540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.407660 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.408229 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:35.408250 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:35.410772 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:35.414353 (+ 3581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.414362 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.414368 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.414943 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.415315 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.415319 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.415322 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.415463 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.416107 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:35.416114 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:35.416320 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 09:17:35.416491 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 09:17:35.416769 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":391,"threads_started":1}
I20260811 09:17:35.417219 7769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.401330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59574 (local address 127.17.16.126:46851)
0811 09:17:35.401506 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.401512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.402286 (+ 774us) server_negotiation.cc:407] Connection header received
0811 09:17:35.402556 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.402561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.402654 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.402726 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.403926 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.404641 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.406695 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.407361 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.407494 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.410964 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.410972 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.410977 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.411021 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.414208 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.415041 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.415045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.415047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.415192 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.415574 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.415578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.415581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.415893 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.416033 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.416629 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.416842 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 09:17:35.418632 7990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:17:35 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
Aug 11 09:17:35 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439855, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:17:35.431373 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.416065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59582 (local address 127.17.16.126:46851)
0811 09:17:35.416533 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.416537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.416556 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:17:35.416598 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.416601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.416651 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.416738 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.418197 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.419117 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.420686 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.421339 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.421458 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.425697 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.425702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.425705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.425733 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.428721 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.429440 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.429445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.429449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.429581 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.429966 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.429969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.429971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.430280 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.430514 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.430829 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.431046 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":232,"threads_started":1}
I20260811 09:17:35.431968 7996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.419341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:36951 (local address 127.0.0.1:56586)
0811 09:17:35.419509 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.419537 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:35.419634 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:35.419899 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:35.419904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:35.420316 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:35.420486 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.420497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.421762 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.421767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.423035 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.423051 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.423719 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.423725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.423900 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.424368 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:35.424383 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:35.426376 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:35.429626 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.429631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.429634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.430082 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.430453 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.430457 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.430461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.430631 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.431253 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:35.431256 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:35.431381 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 09:17:35.431500 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 09:17:35.431740 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 09:17:35.432693 7879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.419383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56586 (local address 127.17.16.125:36951)
0811 09:17:35.419545 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.419549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.419581 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:17:35.419732 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.419735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.419789 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.419858 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.420612 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.421622 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.423170 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.423581 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.423678 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.426573 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.426580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.426585 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.426625 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.429502 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.430185 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.430191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.430194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.430357 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.430740 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.430744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.430747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.431061 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.431203 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.432227 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.432421 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:17:35.433655 7990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:17:35 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439855, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
Aug 11 09:17:35 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:17:35.446640 7996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.434426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45410)
0811 09:17:35.434604 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.434619 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:35.434704 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:35.434994 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:35.434998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:35.435330 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:35.435445 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.435452 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.436444 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.436450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.437313 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.437325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.437981 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.437987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.438177 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.438623 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:35.438638 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:35.440524 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:35.443881 (+ 3357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.443886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.443890 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.444397 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.444947 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.444952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.444956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.445112 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.445710 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:35.445715 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:35.445864 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 09:17:35.446004 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 09:17:35.446293 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 09:17:35.447120 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.434669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.17.16.124:40363)
0811 09:17:35.434795 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.434800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.434814 (+ 14us) server_negotiation.cc:407] Connection header received
0811 09:17:35.434849 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.434851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.434894 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.434964 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.435572 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.436333 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.437433 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.437838 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.437937 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.440695 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.440702 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.440706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.440741 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.443730 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.444534 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.444540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.444544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.444784 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.445227 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.445232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.445235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.445589 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.445748 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.446149 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.446516 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":31}
I20260811 09:17:35.448371 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.432949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.17.16.125:36951)
0811 09:17:35.433519 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.433524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.433544 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:17:35.433589 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.433593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.433655 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.433755 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.435152 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.435956 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.437185 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.437579 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.437683 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.441557 (+ 3874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.441562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.441565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.441591 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.444414 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.445823 (+ 1409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.445828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.445832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.445980 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.446433 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.446438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.446442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.446909 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.447160 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.447653 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.447868 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":239,"threads_started":1}
I20260811 09:17:35.453290 7990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4: Bootstrap starting.
I20260811 09:17:35.454034 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:17:35.456437 7990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:35.457448 7990 log.cc:824] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:35.459853 7990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4: No bootstrap required, opened a new log
I20260811 09:17:35.470228 7990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } }
I20260811 09:17:35.470659 7990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:35.470783 7990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4c90f403a543ecbb64e86764d9e4c4, State: Initialized, Role: FOLLOWER
I20260811 09:17:35.471428 7990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } }
I20260811 09:17:35.478788 8011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } } }
I20260811 09:17:35.479298 8011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:17:35.480033 7990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:17:35.484231 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.462065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45424 (local address 127.17.16.124:40363)
0811 09:17:35.462418 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.462423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.464106 (+ 1683us) server_negotiation.cc:407] Connection header received
0811 09:17:35.464595 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.464601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.464680 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.464761 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.467936 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.469058 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.471478 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.472182 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.472274 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.476351 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.476358 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.476363 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.476402 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.480470 (+ 4068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.481355 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.481360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.481364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.481551 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.481976 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.481982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.481985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.482338 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.482840 (+ 502us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.483430 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.483635 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
I20260811 09:17:35.485620 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.461846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56606 (local address 127.17.16.125:36951)
0811 09:17:35.462519 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.462525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.467643 (+ 5118us) server_negotiation.cc:407] Connection header received
0811 09:17:35.467892 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.467898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.467968 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.468052 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.470568 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.471520 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.473607 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.474222 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.474314 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.477721 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.477728 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.477733 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.477771 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.481777 (+ 4006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.482627 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.482632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.482644 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.482799 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.483232 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.483238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.483243 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.483599 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.483838 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.485041 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.485279 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521}
I20260811 09:17:35.488225 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.461902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59596 (local address 127.17.16.126:46851)
0811 09:17:35.462269 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.462275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.469951 (+ 7676us) server_negotiation.cc:407] Connection header received
0811 09:17:35.470279 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.470284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.470363 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.470434 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.473365 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.474454 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.476747 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.477444 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.477536 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.481743 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.481749 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.481754 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.481793 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.485273 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.486078 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.486084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.486087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.486238 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.486716 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.486721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.486724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.487088 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.487302 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.487749 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.487962 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
W20260811 09:17:35.495385 8033 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:17:35.495564 8033 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:17:35.492579 7882 master_runner.cc:186] Error getting master registration for 127.17.16.126:46851: OK, instance_id { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" instance_seqno: 1786439854392768 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:17:35.503705 7882 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:17:35.527518 7764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } attempt: 4
Aug 11 09:17:35 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
I20260811 09:17:35.542408 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.528012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45440)
0811 09:17:35.528212 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.528232 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:35.528341 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:35.528633 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:35.528638 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:35.529208 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:35.529430 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.529442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.530497 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.530503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.531968 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.531985 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.532730 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.532736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.532951 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.534116 (+ 1165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:35.534150 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:35.536404 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:35.540158 (+ 3754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.540162 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.540165 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.540547 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.540965 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.540968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.540970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.541104 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.541676 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:35.541679 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:35.541807 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 09:17:35.541911 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 09:17:35.542100 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 09:17:35.542685 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.528132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45440 (local address 127.17.16.124:40363)
0811 09:17:35.528280 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.528285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.528302 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:17:35.528445 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.528448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.528503 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.528575 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.529565 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.530366 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.532113 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.532574 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.532706 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.536581 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.536587 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.536592 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.536635 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.540032 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.540661 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.540667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.540670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.540836 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.541210 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.541216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.541220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.541578 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.541728 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.542053 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.542289 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:17:35.548923 7764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858: Bootstrap starting.
I20260811 09:17:35.552867 7764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:35.554175 7764 log.cc:824] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:35.557341 7764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858: No bootstrap required, opened a new log
I20260811 09:17:35.568060 7764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } }
I20260811 09:17:35.568485 7764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:35.568608 7764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150b8d13ac5e45aa8b6b633fa0c74858, State: Initialized, Role: FOLLOWER
I20260811 09:17:35.569273 7764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } }
I20260811 09:17:35.570273 8036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } } }
I20260811 09:17:35.570693 8036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:17:35.571548 7764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:17:35.578423 8036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:17:35.586011 8036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } }
W20260811 09:17:35.590166 8056 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:17:35.590340 8056 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:17:35.592659 8036 leader_election.cc:288] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b75da6945d4863b244cdf12b4237cd (127.17.16.125:36951), 1c4c90f403a543ecbb64e86764d9e4c4 (127.17.16.124:40363)
I20260811 09:17:35.593617 7952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" is_pre_election: true
I20260811 09:17:35.594214 7952 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 150b8d13ac5e45aa8b6b633fa0c74858 in term 0.
I20260811 09:17:35.594810 7695 leader_election.cc:302] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 150b8d13ac5e45aa8b6b633fa0c74858, 1c4c90f403a543ecbb64e86764d9e4c4; no voters:
I20260811 09:17:35.595890 8036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:17:35.596040 8036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:17:35.596140 8036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:35.596570 7874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } attempt: 5
I20260811 09:17:35.599540 8036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } }
I20260811 09:17:35.601402 7952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4c90f403a543ecbb64e86764d9e4c4"
I20260811 09:17:35.601608 7952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:35.605134 7952 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 150b8d13ac5e45aa8b6b633fa0c74858 in term 1.
I20260811 09:17:35.605773 7695 leader_election.cc:302] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 150b8d13ac5e45aa8b6b633fa0c74858, 1c4c90f403a543ecbb64e86764d9e4c4; no voters:
Aug 11 09:17:35 dist-test-slave-ncp9 krb5kdc[7645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439854, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
I20260811 09:17:35.611703 8036 leader_election.cc:288] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [CANDIDATE]: Term 1 election: Requested vote from peers a2b75da6945d4863b244cdf12b4237cd (127.17.16.125:36951), 1c4c90f403a543ecbb64e86764d9e4c4 (127.17.16.124:40363)
I20260811 09:17:35.615711 8036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:17:35.616127 8036 raft_consensus.cc:686] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [term 1 LEADER]: Becoming Leader. State: Replica: 150b8d13ac5e45aa8b6b633fa0c74858, State: Running, Role: LEADER
W20260811 09:17:35.619433 7917 tablet.cc:2369] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:17:35.619896 8036 consensus_queue.cc:237] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } }
I20260811 09:17:35.621446 7877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.597079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40363 (local address 127.0.0.1:45454)
0811 09:17:35.597409 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:35.597429 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:35.597534 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:35.598379 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:35.598385 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:35.598914 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:35.599115 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.599127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.601256 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.601262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.602690 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:35.602708 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.606524 (+ 3816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.606531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:35.606753 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.609271 (+ 2518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:35.609295 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:35.613046 (+ 3751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:35.617392 (+ 4346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.617398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.617402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.617945 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.618385 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:35.618390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:35.618393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:35.618561 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:35.620408 (+ 1847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:35.620413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:35.620572 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 09:17:35.620732 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 09:17:35.621069 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212}
I20260811 09:17:35.622484 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:35.597738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.17.16.124:40363)
0811 09:17:35.597900 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:35.597906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:35.597924 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:17:35.597983 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:35.597988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:35.598059 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:35.598148 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:35.599285 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.600386 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.605396 (+ 5010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:35.606130 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:35.606258 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:35.613263 (+ 7005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:35.613270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:35.613275 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:35.613318 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:35.617223 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.618087 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.618092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.618095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.618247 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:35.618712 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:35.618717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:35.618720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:35.619094 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:35.621700 (+ 2606us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:35.621857 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 09:17:35.622101 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:17:35.600605 7671 master_runner.cc:186] Error getting master registration for 127.17.16.125:36951: OK, instance_id { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" instance_seqno: 1786439854874941 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:17:35.623755 7671 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:17:35.625097 8058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150b8d13ac5e45aa8b6b633fa0c74858. Latest consensus state: current_term: 1 leader_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } } }
I20260811 09:17:35.625494 8058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:35.628271 7874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd: Bootstrap starting.
I20260811 09:17:35.632386 7874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd: Neither blocks nor log segments found. Creating new log.
W20260811 09:17:35.633212 7704 tablet.cc:2369] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:17:35.633749 7874 log.cc:824] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:35.639325 7874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd: No bootstrap required, opened a new log
I20260811 09:17:35.645428 8061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:17:35.647415 8061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:17:35.658716 7874 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } }
I20260811 09:17:35.659305 7874 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:35.659487 7874 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b75da6945d4863b244cdf12b4237cd, State: Initialized, Role: FOLLOWER
I20260811 09:17:35.660498 7874 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } }
I20260811 09:17:35.665452 8065 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } } }
I20260811 09:17:35.665915 8065 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:17:35.667845 7874 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:17:35.667934 7839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b75da6945d4863b244cdf12b4237cd"
I20260811 09:17:35.667894 7840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b75da6945d4863b244cdf12b4237cd" is_pre_election: true
I20260811 09:17:35.668453 7839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:35.671923 7839 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 150b8d13ac5e45aa8b6b633fa0c74858 in term 1.
I20260811 09:17:35.676604 7952 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [term 1 FOLLOWER]: Refusing update from remote peer 150b8d13ac5e45aa8b6b633fa0c74858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:17:35.677692 8036 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:17:35.684775 7839 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [term 1 FOLLOWER]: Refusing update from remote peer 150b8d13ac5e45aa8b6b633fa0c74858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:17:35.685503 8036 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:17:35.723012 8065 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150b8d13ac5e45aa8b6b633fa0c74858. Latest consensus state: current_term: 1 leader_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } } }
I20260811 09:17:35.723410 8065 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:17:35.724002 8064 mvcc.cc:205] Tried to move back new op lower bound from 7317257648838643712 to 7317257648624472064. Current Snapshot: MvccSnapshot[applied={T|T < 7317257648838643712}]
I20260811 09:17:35.729396 8036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } } }
I20260811 09:17:35.729815 8036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:35.730054 8069 mvcc.cc:205] Tried to move back new op lower bound from 7317257648838643712 to 7317257648624472064. Current Snapshot: MvccSnapshot[applied={T|T < 7317257648838643712}]
I20260811 09:17:35.735002 8065 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } } }
I20260811 09:17:35.735248 8065 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:17:35.741632 8061 catalog_manager.cc:1347] Generated new cluster ID: f63e540b293b4ad486cf0ae5b58b4dff
I20260811 09:17:35.741746 8061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:17:35.741792 8011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150b8d13ac5e45aa8b6b633fa0c74858. Latest consensus state: current_term: 1 leader_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } } }
I20260811 09:17:35.742093 8011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:17:35.744688 8058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } } }
I20260811 09:17:35.744984 8058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858 [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:35.748469 8011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "150b8d13ac5e45aa8b6b633fa0c74858" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 46851 } } peers { permanent_uuid: "a2b75da6945d4863b244cdf12b4237cd" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 36951 } } peers { permanent_uuid: "1c4c90f403a543ecbb64e86764d9e4c4" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40363 } } }
I20260811 09:17:35.748787 8011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:17:35.776944 8061 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:17:35.779014 8061 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:17:35.805539 8061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 150b8d13ac5e45aa8b6b633fa0c74858: Generated new TSK 0
I20260811 09:17:35.806702 8061 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:17:35.850982 8092 catalog_manager.cc:1259] Loaded cluster ID: f63e540b293b4ad486cf0ae5b58b4dff
I20260811 09:17:35.851091 8092 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd: loading cluster ID for follower catalog manager: success
I20260811 09:17:35.854784 8092 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd: acquiring CA information for follower catalog manager: success
I20260811 09:17:35.857170 8092 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a2b75da6945d4863b244cdf12b4237cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:17:36.497622 8033 catalog_manager.cc:1259] Loaded cluster ID: f63e540b293b4ad486cf0ae5b58b4dff
I20260811 09:17:36.497723 8033 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4: loading cluster ID for follower catalog manager: success
I20260811 09:17:36.500088 8033 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4: acquiring CA information for follower catalog manager: success
I20260811 09:17:36.501871 8033 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1c4c90f403a543ecbb64e86764d9e4c4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:17:36.971228 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 7671
I20260811 09:17:36.973222 7757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:37.083603 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 7671
I20260811 09:17:37.114210 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 7774
I20260811 09:17:37.115335 7869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:37.219628 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 7774
I20260811 09:17:37.243355 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 7882
I20260811 09:17:37.244545 7984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:37.350670 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 7882
2026-08-11T09:17:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4901 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:17:37 dist-test-slave-ncp9 krb5kdc[8108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:17:37 dist-test-slave-ncp9 krb5kdc[8108](info): set up 2 sockets
Aug 11 09:17:37 dist-test-slave-ncp9 krb5kdc[8108](info): commencing operation
krb5kdc: starting...
W20260811 09:17:38.584458 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:17:38 dist-test-slave-ncp9 krb5kdc[8108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:17:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:17:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:17:38.855942 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:33115
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:33115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:17:39.186157 8129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:39.186434 8129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:39.186575 8129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:39.193641 8129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:17:39.193743 8129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:39.193815 8129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:39.193886 8129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:17:39.193945 8129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:17:39.203620 8129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:33115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:33115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:39.204882 8129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:39.206154 8129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:39.217612 8135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:39.217434 8136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:39.217478 8139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:39.218684 8129 server_base.cc:1034] running on GCE node
I20260811 09:17:39.219395 8129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:39.220697 8129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:39.221907 8129 hybrid_clock.cc:648] HybridClock initialized: now 1786439859221833 us; error 58 us; skew 500 ppm
I20260811 09:17:39.222254 8129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:39 dist-test-slave-ncp9 krb5kdc[8108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439859, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:39.225226 8129 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:17:39.226617 8129 webserver.cc:460] Webserver started at http://127.17.16.126:40419/ using document root <none> and password file <none>
I20260811 09:17:39.227233 8129 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:39.227349 8129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:39.227640 8129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:17:39.230567 8129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "21cc8a1e1727435cb7747361c2e42b01"
format_stamp: "Formatted at 2026-08-11 09:17:39 on dist-test-slave-ncp9"
server_key: "feb55fcaff65a59a25f87bd8b95a25b7"
server_key_iv: "aa6214e0e1a37ce681b7f37ef04ed157"
server_key_version: "encryptionkey@0"
I20260811 09:17:39.231280 8129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "21cc8a1e1727435cb7747361c2e42b01"
format_stamp: "Formatted at 2026-08-11 09:17:39 on dist-test-slave-ncp9"
server_key: "feb55fcaff65a59a25f87bd8b95a25b7"
server_key_iv: "aa6214e0e1a37ce681b7f37ef04ed157"
server_key_version: "encryptionkey@0"
I20260811 09:17:39.236146 8129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:17:39.239356 8151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:39.240564 8129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 09:17:39.240722 8129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "21cc8a1e1727435cb7747361c2e42b01"
format_stamp: "Formatted at 2026-08-11 09:17:39 on dist-test-slave-ncp9"
server_key: "feb55fcaff65a59a25f87bd8b95a25b7"
server_key_iv: "aa6214e0e1a37ce681b7f37ef04ed157"
server_key_version: "encryptionkey@0"
I20260811 09:17:39.240913 8129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:39.253749 8129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:39.257339 8129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:39.257594 8129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:39.276125 8129 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:33115
I20260811 09:17:39.276168 8232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:33115 every 8 connection(s)
I20260811 09:17:39.277506 8129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:17:39.281286 8234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:39.281416 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 8129
I20260811 09:17:39.281639 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:17:39.282541 17473 external_mini_cluster.cc:1428] Setting key d49f75e0d54f8fb00fd251f293700f9d
I20260811 09:17:39.293571 8234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01: Bootstrap starting.
Aug 11 09:17:39 dist-test-slave-ncp9 krb5kdc[8108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:39.296646 8234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:39.297685 8234 log.cc:824] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:39.300361 8234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01: No bootstrap required, opened a new log
I20260811 09:17:39.304682 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:39.284466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60086 (local address 127.17.16.126:33115)
0811 09:17:39.285273 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:39.285284 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:39.285325 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:17:39.287554 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:39.287582 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:39.287973 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:39.288323 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:39.289882 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:39.291252 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:39.292593 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:39.293320 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:39.293502 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:39.297763 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:39.297773 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:39.297792 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:39.297842 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:39.300773 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:39.301487 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:39.301494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:39.301500 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:39.301658 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:39.302052 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:39.302056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:39.302058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:39.302580 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:39.302887 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:39.303353 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 09:17:39.303735 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":608,"thread_start_us":362,"threads_started":1}
I20260811 09:17:39.308312 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:17:39.308241 8234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cc8a1e1727435cb7747361c2e42b01" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33115 } }
I20260811 09:17:39.308626 8234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:39.308717 8234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21cc8a1e1727435cb7747361c2e42b01, State: Initialized, Role: FOLLOWER
I20260811 09:17:39.309274 8234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cc8a1e1727435cb7747361c2e42b01" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33115 } }
I20260811 09:17:39.309489 8234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:17:39.309621 8234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:17:39.309759 8234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:39.312063 8234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cc8a1e1727435cb7747361c2e42b01" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33115 } }
I20260811 09:17:39.312500 8234 leader_election.cc:302] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21cc8a1e1727435cb7747361c2e42b01; no voters:
I20260811 09:17:39.313059 8234 leader_election.cc:288] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:17:39.313160 8239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:17:39.313795 8239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [term 1 LEADER]: Becoming Leader. State: Replica: 21cc8a1e1727435cb7747361c2e42b01, State: Running, Role: LEADER
I20260811 09:17:39.314292 8239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cc8a1e1727435cb7747361c2e42b01" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33115 } }
I20260811 09:17:39.314935 8234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:17:39.316861 8241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21cc8a1e1727435cb7747361c2e42b01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cc8a1e1727435cb7747361c2e42b01" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33115 } } }
I20260811 09:17:39.317363 8241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:39.316936 8245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21cc8a1e1727435cb7747361c2e42b01. Latest consensus state: current_term: 1 leader_uuid: "21cc8a1e1727435cb7747361c2e42b01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21cc8a1e1727435cb7747361c2e42b01" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 33115 } } }
I20260811 09:17:39.318056 8245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01 [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:39.318190 8251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:17:39.322711 8251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:17:39.330497 8251 catalog_manager.cc:1347] Generated new cluster ID: 8d6bbbc7336a45d2be8893a39898692b
I20260811 09:17:39.330613 8251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:17:39.341445 8251 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:17:39.342630 8251 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:17:39.347918 8251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21cc8a1e1727435cb7747361c2e42b01: Generated new TSK 0
I20260811 09:17:39.348594 8251 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:17:39.368794 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 8129
I20260811 09:17:39.369910 8228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:39.460399 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 8129
2026-08-11T09:17:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2105 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:17:39 dist-test-slave-ncp9 krb5kdc[8275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:17:39 dist-test-slave-ncp9 krb5kdc[8275](info): set up 2 sockets
Aug 11 09:17:39 dist-test-slave-ncp9 krb5kdc[8275](info): commencing operation
krb5kdc: starting...
W20260811 09:17:40.716797 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:17:40 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:17:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:17:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:17:40.980188 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:37471
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:37471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:17:41.304997 8302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:41.305266 8302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:41.305361 8302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:41.312337 8302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:17:41.312434 8302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:41.312484 8302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:41.312539 8302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:17:41.312582 8302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:17:41.321848 8302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:37471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:37471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:41.323045 8302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:41.324244 8302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:41.335052 8309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:41.335103 8313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:41.335120 8316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:41.335844 8302 server_base.cc:1034] running on GCE node
I20260811 09:17:41.336524 8302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:41.337796 8302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:41.338989 8302 hybrid_clock.cc:648] HybridClock initialized: now 1786439861338935 us; error 49 us; skew 500 ppm
I20260811 09:17:41.339282 8302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:41 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439861, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:41.342275 8302 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:17:41.343595 8302 webserver.cc:460] Webserver started at http://127.17.16.126:46467/ using document root <none> and password file <none>
I20260811 09:17:41.344182 8302 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:41.344277 8302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:41.344512 8302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:17:41.347287 8302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "d1d4e62e0d3041138b484db176bb393c"
format_stamp: "Formatted at 2026-08-11 09:17:41 on dist-test-slave-ncp9"
server_key: "9773a101ac607923977c8e6a9ce12ee9"
server_key_iv: "562fc26efedcbc1a51eba2a56ed9ea53"
server_key_version: "encryptionkey@0"
I20260811 09:17:41.347962 8302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "d1d4e62e0d3041138b484db176bb393c"
format_stamp: "Formatted at 2026-08-11 09:17:41 on dist-test-slave-ncp9"
server_key: "9773a101ac607923977c8e6a9ce12ee9"
server_key_iv: "562fc26efedcbc1a51eba2a56ed9ea53"
server_key_version: "encryptionkey@0"
I20260811 09:17:41.352805 8302 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:17:41.356151 8327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:41.357455 8302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:17:41.357595 8302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "d1d4e62e0d3041138b484db176bb393c"
format_stamp: "Formatted at 2026-08-11 09:17:41 on dist-test-slave-ncp9"
server_key: "9773a101ac607923977c8e6a9ce12ee9"
server_key_iv: "562fc26efedcbc1a51eba2a56ed9ea53"
server_key_version: "encryptionkey@0"
I20260811 09:17:41.357769 8302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:41.381321 8302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:41.385321 8302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:41.385550 8302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:41.402506 8302 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:37471
I20260811 09:17:41.402529 8405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:37471 every 8 connection(s)
I20260811 09:17:41.403810 8302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:17:41.404528 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 8302
I20260811 09:17:41.404758 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:17:41.405504 17473 external_mini_cluster.cc:1428] Setting key bd598b2b864a5309bd56a440b6cb04c3
I20260811 09:17:41.407913 8406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:17:41 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:41.420660 8406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Bootstrap starting.
I20260811 09:17:41.423684 8406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:41.424642 8406 log.cc:824] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:41.427366 8406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: No bootstrap required, opened a new log
I20260811 09:17:41.427630 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:41.407393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45924 (local address 127.17.16.126:37471)
0811 09:17:41.408153 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:41.408167 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:41.408207 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:17:41.409843 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:41.409871 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:41.410375 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:41.410816 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:41.412515 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:41.413882 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:41.415651 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:41.416273 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:41.416395 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:41.420993 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:41.421002 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:41.421015 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:41.421052 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:41.423840 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:41.424482 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:41.424492 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:41.424498 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:41.424673 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:41.425065 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:41.425069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:41.425071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:41.425590 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:41.425878 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:41.426342 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 09:17:41.426724 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":319,"threads_started":1}
I20260811 09:17:41.435449 8406 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:41.435796 8406 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:41.435883 8406 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d4e62e0d3041138b484db176bb393c, State: Initialized, Role: FOLLOWER
I20260811 09:17:41.436390 8406 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:41.436600 8406 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:17:41.436728 8406 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:17:41.436882 8406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:41.439316 8406 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:41.439764 8406 leader_election.cc:302] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1d4e62e0d3041138b484db176bb393c; no voters:
I20260811 09:17:41.440294 8406 leader_election.cc:288] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:17:41.440400 8411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:17:41.441089 8411 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 1 LEADER]: Becoming Leader. State: Replica: d1d4e62e0d3041138b484db176bb393c, State: Running, Role: LEADER
I20260811 09:17:41.441632 8411 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:41.442281 8406 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:17:41.444954 8412 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1d4e62e0d3041138b484db176bb393c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } } }
I20260811 09:17:41.446126 8412 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:41.445348 8414 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1d4e62e0d3041138b484db176bb393c. Latest consensus state: current_term: 1 leader_uuid: "d1d4e62e0d3041138b484db176bb393c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } } }
I20260811 09:17:41.446514 8414 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:41.449484 8419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:17:41.455330 8419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:17:41.463896 8419 catalog_manager.cc:1347] Generated new cluster ID: c39d92db02f14c958cfc1923f0df2bcf
I20260811 09:17:41.464031 8419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:17:41.476475 8419 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:17:41.477746 8419 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:17:41.484829 8419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Generated new TSK 0
I20260811 09:17:41.485643 8419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:17:41.556226 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37471
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:17:41.888825 8439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:41.889154 8439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:41.889266 8439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:41.896517 8439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:41.896631 8439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:41.896757 8439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:17:41.906581 8439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:41.907794 8439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:41.909130 8439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:41.922614 8451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:41.922678 8455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:41.922698 8452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:41.923316 8439 server_base.cc:1034] running on GCE node
I20260811 09:17:41.923916 8439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:41.924750 8439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:41.925958 8439 hybrid_clock.cc:648] HybridClock initialized: now 1786439861925906 us; error 46 us; skew 500 ppm
I20260811 09:17:41.926250 8439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:41 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439861, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:41.930007 8439 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:17:41.931710 8439 webserver.cc:460] Webserver started at http://127.17.16.65:43595/ using document root <none> and password file <none>
I20260811 09:17:41.932356 8439 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:41.932456 8439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:41.932710 8439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:17:41.935540 8439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "c007f901f0684f17a8b47286b2538651"
format_stamp: "Formatted at 2026-08-11 09:17:41 on dist-test-slave-ncp9"
server_key: "16f777f49925bfe1b4d48d653e1e4a4d"
server_key_iv: "c103ff9c38c1538dabbfb2520f609141"
server_key_version: "encryptionkey@0"
I20260811 09:17:41.936347 8439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "c007f901f0684f17a8b47286b2538651"
format_stamp: "Formatted at 2026-08-11 09:17:41 on dist-test-slave-ncp9"
server_key: "16f777f49925bfe1b4d48d653e1e4a4d"
server_key_iv: "c103ff9c38c1538dabbfb2520f609141"
server_key_version: "encryptionkey@0"
I20260811 09:17:41.941514 8439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:17:41.945034 8467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:41.946298 8439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:17:41.946445 8439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "c007f901f0684f17a8b47286b2538651"
format_stamp: "Formatted at 2026-08-11 09:17:41 on dist-test-slave-ncp9"
server_key: "16f777f49925bfe1b4d48d653e1e4a4d"
server_key_iv: "c103ff9c38c1538dabbfb2520f609141"
server_key_version: "encryptionkey@0"
I20260811 09:17:41.946632 8439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:41.965746 8439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:41.969555 8439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:41.969815 8439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:41.970741 8439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:41.972056 8439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:41.972152 8439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:41.972285 8439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:41.972339 8439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:42.002924 8439 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:45591
I20260811 09:17:42.002967 8634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:45591 every 8 connection(s)
I20260811 09:17:42.004292 8439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:17:42.013079 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 8439
I20260811 09:17:42.013303 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:17:42.013947 17473 external_mini_cluster.cc:1428] Setting key 3cdd5ddeb30f95cb9efea74f14346067
Aug 11 09:17:42 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439861, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:42.023826 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:42.006784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:50729 (local address 127.17.16.126:37471)
0811 09:17:42.007221 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:42.007226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:42.008181 (+ 955us) server_negotiation.cc:407] Connection header received
0811 09:17:42.009988 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:42.009994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:42.010071 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:42.010180 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:42.011627 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:42.012408 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:42.014125 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:42.014668 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:42.014797 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:42.018096 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:42.018103 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:42.018108 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:42.018147 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:42.021009 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:42.021680 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:42.021685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:42.021688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:42.021834 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:42.022214 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:42.022219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:42.022222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:42.022530 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:42.022666 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:42.023296 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 09:17:42.023499 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":224,"threads_started":1}
I20260811 09:17:42.024686 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:42.007289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:50729)
0811 09:17:42.008009 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:42.008055 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:42.009726 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:42.010414 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:42.010422 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:42.010910 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:42.011455 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:42.011472 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:42.012549 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:42.012553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:42.013949 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:42.013962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:42.014897 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:42.014902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:42.015097 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:42.015768 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:42.015795 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:42.017874 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:42.021141 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:42.021147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:42.021159 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:42.021556 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:42.021922 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:42.021925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:42.021928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:42.022097 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:42.022632 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:42.022646 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:42.022936 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 09:17:42.023155 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 09:17:42.023525 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":289,"threads_started":1}
I20260811 09:17:42.026170 8635 heartbeater.cc:347] Connected to a master server at 127.17.16.126:37471
I20260811 09:17:42.026441 8635 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:42.027127 8635 heartbeater.cc:511] Master 127.17.16.126:37471 requested a full tablet report, sending...
I20260811 09:17:42.029189 8350 ts_manager.cc:194] Registered new tserver with Master: c007f901f0684f17a8b47286b2538651 (127.17.16.65:45591)
I20260811 09:17:42.030583 8350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:50729
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:17:42.071574 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37471
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:17:42.396342 8643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:42.396624 8643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:42.396767 8643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:42.403908 8643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:42.404018 8643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:42.404168 8643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:17:42.413525 8643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:42.414691 8643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:42.415920 8643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:42.428273 8650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:42.428333 8655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:42.428494 8658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:42.429288 8643 server_base.cc:1034] running on GCE node
I20260811 09:17:42.429844 8643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:42.430682 8643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:42.431878 8643 hybrid_clock.cc:648] HybridClock initialized: now 1786439862431834 us; error 31 us; skew 500 ppm
I20260811 09:17:42.432175 8643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:42 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:42.435065 8643 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:17:42.436431 8643 webserver.cc:460] Webserver started at http://127.17.16.66:46291/ using document root <none> and password file <none>
I20260811 09:17:42.437093 8643 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:42.437212 8643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:42.437497 8643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:17:42.440217 8643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "be1b77a6ada0496f96ea1cbe1ef17f42"
format_stamp: "Formatted at 2026-08-11 09:17:42 on dist-test-slave-ncp9"
server_key: "2356194807b09d431df3655bf32de647"
server_key_iv: "d07fda8a4c287fa5f360bd62bbc1f0d9"
server_key_version: "encryptionkey@0"
I20260811 09:17:42.440963 8643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "be1b77a6ada0496f96ea1cbe1ef17f42"
format_stamp: "Formatted at 2026-08-11 09:17:42 on dist-test-slave-ncp9"
server_key: "2356194807b09d431df3655bf32de647"
server_key_iv: "d07fda8a4c287fa5f360bd62bbc1f0d9"
server_key_version: "encryptionkey@0"
I20260811 09:17:42.445936 8643 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 09:17:42.449260 8668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:42.450505 8643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:17:42.450685 8643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "be1b77a6ada0496f96ea1cbe1ef17f42"
format_stamp: "Formatted at 2026-08-11 09:17:42 on dist-test-slave-ncp9"
server_key: "2356194807b09d431df3655bf32de647"
server_key_iv: "d07fda8a4c287fa5f360bd62bbc1f0d9"
server_key_version: "encryptionkey@0"
I20260811 09:17:42.450856 8643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:42.477672 8643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:42.481424 8643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:42.481678 8643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:42.482568 8643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:42.483889 8643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:42.483960 8643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:42.484057 8643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:42.484105 8643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:42.513958 8643 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:35133
I20260811 09:17:42.514008 8833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:35133 every 8 connection(s)
I20260811 09:17:42.515202 8643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:17:42.516858 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 8643
I20260811 09:17:42.517081 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:17:42.517562 17473 external_mini_cluster.cc:1428] Setting key 097c33622d9ab76937d94f71d907cc6d
Aug 11 09:17:42 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:42.535029 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:42.517959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:53991 (local address 127.17.16.126:37471)
0811 09:17:42.518174 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:42.518180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:42.519470 (+ 1290us) server_negotiation.cc:407] Connection header received
0811 09:17:42.521343 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:42.521348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:42.521427 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:42.521527 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:42.522902 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:42.523855 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:42.525374 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:42.525945 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:42.526073 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:42.529413 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:42.529420 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:42.529424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:42.529462 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:42.532150 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:42.532972 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:42.532976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:42.532978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:42.533124 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:42.533543 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:42.533546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:42.533547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:42.533778 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:42.533880 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:42.534620 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 09:17:42.534787 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 09:17:42.536372 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:42.518514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:53991)
0811 09:17:42.519329 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:42.519377 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:42.521090 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:42.521695 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:42.521703 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:42.522197 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:42.522737 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:42.522755 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:42.523973 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:42.523977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:42.525242 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:42.525255 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:42.526153 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:42.526158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:42.526352 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:42.526963 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:42.526985 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:42.529200 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:42.532288 (+ 3088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:42.532297 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:42.532313 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:42.532843 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:42.533225 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:42.533229 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:42.533232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:42.533450 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:42.533896 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:42.533903 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:42.534274 (+ 371us) client_negotiation.cc:769] Sending connection context
0811 09:17:42.534558 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 09:17:42.535072 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":289,"threads_started":1}
I20260811 09:17:42.537771 8836 heartbeater.cc:347] Connected to a master server at 127.17.16.126:37471
I20260811 09:17:42.538062 8836 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:42.538736 8836 heartbeater.cc:511] Master 127.17.16.126:37471 requested a full tablet report, sending...
I20260811 09:17:42.540154 8350 ts_manager.cc:194] Registered new tserver with Master: be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133)
I20260811 09:17:42.540979 8350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:53991
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:17:42.578141 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37471
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:17:42.900288 8846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:42.900571 8846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:42.900710 8846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:42.907650 8846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:42.907763 8846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:42.907922 8846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:17:42.917513 8846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:42.918727 8846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:42.920032 8846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:42.932449 8854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:42.932484 8855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:42.932533 8860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:42.933477 8846 server_base.cc:1034] running on GCE node
I20260811 09:17:42.934180 8846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:42.935135 8846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:42.936329 8846 hybrid_clock.cc:648] HybridClock initialized: now 1786439862936282 us; error 44 us; skew 500 ppm
I20260811 09:17:42.936621 8846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:42 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:42.939531 8846 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:17:42.940824 8846 webserver.cc:460] Webserver started at http://127.17.16.67:40235/ using document root <none> and password file <none>
I20260811 09:17:42.941591 8846 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:42.941699 8846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:42.941973 8846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:17:42.944864 8846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "bf2c040634154395bc16c3879da07551"
format_stamp: "Formatted at 2026-08-11 09:17:42 on dist-test-slave-ncp9"
server_key: "cf2a47a5d6657ae872ae22931f59c31e"
server_key_iv: "a939ba571e330ced749dd005a0fc3a22"
server_key_version: "encryptionkey@0"
I20260811 09:17:42.945664 8846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "bf2c040634154395bc16c3879da07551"
format_stamp: "Formatted at 2026-08-11 09:17:42 on dist-test-slave-ncp9"
server_key: "cf2a47a5d6657ae872ae22931f59c31e"
server_key_iv: "a939ba571e330ced749dd005a0fc3a22"
server_key_version: "encryptionkey@0"
I20260811 09:17:42.950482 8846 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:17:42.953908 8871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:42.955209 8846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:17:42.955349 8846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "bf2c040634154395bc16c3879da07551"
format_stamp: "Formatted at 2026-08-11 09:17:42 on dist-test-slave-ncp9"
server_key: "cf2a47a5d6657ae872ae22931f59c31e"
server_key_iv: "a939ba571e330ced749dd005a0fc3a22"
server_key_version: "encryptionkey@0"
I20260811 09:17:42.955525 8846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:42.980145 8846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:42.983994 8846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:42.984267 8846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:42.985200 8846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:42.986479 8846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:42.986553 8846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:42.986666 8846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:42.986714 8846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:43.016526 8846 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:42909
I20260811 09:17:43.016589 9036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:42909 every 8 connection(s)
I20260811 09:17:43.017732 8846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:17:43.022058 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 8846
I20260811 09:17:43.022264 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:17:43.022694 17473 external_mini_cluster.cc:1428] Setting key e5006d8ffc4f50c2588408b93573e934
Aug 11 09:17:43 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439862, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:43.033815 8635 heartbeater.cc:503] Master 127.17.16.126:37471 was elected leader, sending a full tablet report...
I20260811 09:17:43.036391 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.020145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:56291 (local address 127.17.16.126:37471)
0811 09:17:43.020322 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.020326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.021447 (+ 1121us) server_negotiation.cc:407] Connection header received
0811 09:17:43.023303 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.023309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.023442 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.023539 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:43.025314 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.026079 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.027515 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.027892 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.027978 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.030990 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:43.030994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:43.030998 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:43.031027 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:43.033343 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:43.033976 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:43.033980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:43.033982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:43.034123 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:43.034625 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:43.034630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:43.034633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:43.034971 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:43.035135 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.035861 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.036076 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:17:43.037318 9039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.020576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:56291)
0811 09:17:43.021300 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:43.021342 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:43.023046 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:43.023917 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:43.023924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:43.024392 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:43.025141 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.025167 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.026214 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.026218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.027409 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.027421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.028057 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.028062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.028243 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.028773 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:43.028793 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:43.030820 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:43.033451 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:43.033457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:43.033467 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:43.033848 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:43.034253 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:43.034258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:43.034262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:43.034501 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:43.035099 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:43.035104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:43.035486 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 09:17:43.035728 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 09:17:43.036112 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":298,"threads_started":1}
I20260811 09:17:43.038563 9037 heartbeater.cc:347] Connected to a master server at 127.17.16.126:37471
I20260811 09:17:43.038839 9037 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:43.039508 9037 heartbeater.cc:511] Master 127.17.16.126:37471 requested a full tablet report, sending...
I20260811 09:17:43.040750 8350 ts_manager.cc:194] Registered new tserver with Master: bf2c040634154395bc16c3879da07551 (127.17.16.67:42909)
I20260811 09:17:43.041539 8350 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:56291
I20260811 09:17:43.049806 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:17:43.066071 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.054349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.17.16.126:37471)
0811 09:17:43.054503 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.054507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.054973 (+ 466us) server_negotiation.cc:407] Connection header received
0811 09:17:43.055123 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.055127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.055199 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.055274 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:43.056543 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.057324 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.058618 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.059030 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.059117 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.061520 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:43.061525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:43.061528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:43.061574 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:43.063908 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:43.064481 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:43.064485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:43.064487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:43.064600 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:43.064866 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:43.064868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:43.064870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:43.065217 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:43.065395 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.065634 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.065788 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:17:43.069821 8350 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:17:43.076507 8350 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:17:43.097050 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.088536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52588 (local address 127.17.16.67:42909)
0811 09:17:43.089361 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.089370 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.090037 (+ 667us) server_negotiation.cc:407] Connection header received
0811 09:17:43.090133 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.090140 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.090355 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.090531 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:43.091259 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.092579 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.094800 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.095936 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.096064 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.096287 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.096542 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.096702 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":1403,"threads_started":1}
I20260811 09:17:43.097105 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.087970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:42909 (local address 127.0.0.1:52588)
0811 09:17:43.089775 (+ 1805us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:43.089795 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:43.089919 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:43.090681 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:43.090685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:43.090710 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:43.091023 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.091036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.092724 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.092730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.094600 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.094616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.096116 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.096122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.096307 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.096356 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 09:17:43.096487 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 09:17:43.096674 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1685,"mutex_wait_us":83,"thread_start_us":726,"threads_started":1}
I20260811 09:17:43.097486 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.087881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57372 (local address 127.17.16.66:35133)
0811 09:17:43.088459 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.088468 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.089304 (+ 836us) server_negotiation.cc:407] Connection header received
0811 09:17:43.089405 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.089412 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.089660 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.089850 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:43.090751 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.092248 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.094719 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.096090 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.096218 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.096403 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.097009 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.097172 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":294,"threads_started":1}
I20260811 09:17:43.097898 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.087612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35133 (local address 127.0.0.1:57372)
0811 09:17:43.088867 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:43.088925 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:43.089115 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:43.090142 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:43.090149 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:43.090180 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:43.090560 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.090574 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.092384 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.092389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.094504 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.094519 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.096544 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.096551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.096720 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.096751 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:17:43.096843 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 09:17:43.097588 (+ 745us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"mutex_wait_us":4,"spinlock_wait_cycles":2944,"thread_start_us":1888,"threads_started":1}
I20260811 09:17:43.098311 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.089455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45591 (local address 127.0.0.1:41628)
0811 09:17:43.090074 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:43.090090 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:43.090210 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:43.091293 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:43.091298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:43.091322 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:43.091723 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.091737 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.093653 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.093659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.095757 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.095772 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.097539 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.097546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.097713 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.097745 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:17:43.097818 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 09:17:43.097939 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":2,"spinlock_wait_cycles":33664,"thread_start_us":634,"threads_started":1}
I20260811 09:17:43.098999 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.089805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41628 (local address 127.17.16.65:45591)
0811 09:17:43.090750 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.090758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.090781 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:17:43.090868 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.090876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.091094 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.091899 (+ 805us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:43.092134 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.093508 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.095929 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.097300 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.098104 (+ 804us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.098342 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.098541 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.098702 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":625,"threads_started":1}
I20260811 09:17:43.101229 8742 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92 (DEFAULT_TABLE table=test-table [id=3932e049f62d4bf48811756f427f4e68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:17:43.101241 8940 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92 (DEFAULT_TABLE table=test-table [id=3932e049f62d4bf48811756f427f4e68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:17:43.102398 8742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:43.102651 8940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:43.102246 8545 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92 (DEFAULT_TABLE table=test-table [id=3932e049f62d4bf48811756f427f4e68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:17:43.103480 8545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:43.113116 9065 tablet_bootstrap.cc:492] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: Bootstrap starting.
I20260811 09:17:43.116437 9067 tablet_bootstrap.cc:492] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: Bootstrap starting.
I20260811 09:17:43.117515 9065 tablet_bootstrap.cc:654] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:43.117772 9068 tablet_bootstrap.cc:492] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: Bootstrap starting.
I20260811 09:17:43.118924 9065 log.cc:824] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:43.119824 9067 tablet_bootstrap.cc:654] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:43.120838 9068 tablet_bootstrap.cc:654] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:43.120918 9067 log.cc:824] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:43.121917 9068 log.cc:824] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:43.121981 9065 tablet_bootstrap.cc:492] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: No bootstrap required, opened a new log
I20260811 09:17:43.122241 9065 ts_tablet_manager.cc:1397] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 09:17:43.123862 9067 tablet_bootstrap.cc:492] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: No bootstrap required, opened a new log
I20260811 09:17:43.124106 9067 ts_tablet_manager.cc:1397] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 09:17:43.124230 9068 tablet_bootstrap.cc:492] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: No bootstrap required, opened a new log
I20260811 09:17:43.124450 9068 ts_tablet_manager.cc:1397] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 09:17:43.131548 9068 raft_consensus.cc:348] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } }
I20260811 09:17:43.131863 9068 raft_consensus.cc:374] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:43.131951 9068 raft_consensus.cc:729] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c007f901f0684f17a8b47286b2538651, State: Initialized, Role: FOLLOWER
I20260811 09:17:43.132462 9068 consensus_queue.cc:260] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } }
I20260811 09:17:43.132369 9065 raft_consensus.cc:348] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } }
I20260811 09:17:43.132766 9065 raft_consensus.cc:374] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:43.132859 9065 raft_consensus.cc:729] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be1b77a6ada0496f96ea1cbe1ef17f42, State: Initialized, Role: FOLLOWER
I20260811 09:17:43.133548 9065 consensus_queue.cc:260] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } }
I20260811 09:17:43.134181 9068 ts_tablet_manager.cc:1428] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260811 09:17:43.135053 8836 heartbeater.cc:503] Master 127.17.16.126:37471 was elected leader, sending a full tablet report...
I20260811 09:17:43.135641 9065 ts_tablet_manager.cc:1428] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 09:17:43.135565 9067 raft_consensus.cc:348] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } }
I20260811 09:17:43.135950 9067 raft_consensus.cc:374] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:43.136037 9067 raft_consensus.cc:729] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf2c040634154395bc16c3879da07551, State: Initialized, Role: FOLLOWER
I20260811 09:17:43.137056 9067 consensus_queue.cc:260] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } }
I20260811 09:17:43.138113 9037 heartbeater.cc:503] Master 127.17.16.126:37471 was elected leader, sending a full tablet report...
I20260811 09:17:43.138859 9067 ts_tablet_manager.cc:1428] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
W20260811 09:17:43.256803 8637 tablet.cc:2369] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:17:43.263039 9076 raft_consensus.cc:482] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:17:43.263329 9076 raft_consensus.cc:504] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } }
I20260811 09:17:43.265064 9076 leader_election.cc:288] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf2c040634154395bc16c3879da07551 (127.17.16.67:42909), c007f901f0684f17a8b47286b2538651 (127.17.16.65:45591)
W20260811 09:17:43.267055 8838 tablet.cc:2369] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:17:43.269423 9038 tablet.cc:2369] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:17:43.272168 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.265389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:42909 (local address 127.17.16.66:56637)
0811 09:17:43.266129 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:43.266148 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:43.266296 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:43.266714 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:43.266719 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:43.266755 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:43.267209 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.267224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.268406 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.268411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.270461 (+ 2050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.270474 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.271567 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.271573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.271704 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.271727 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:17:43.271799 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 09:17:43.271884 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":40,"thread_start_us":414,"threads_started":1}
I20260811 09:17:43.272171 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.265487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:56637 (local address 127.17.16.67:42909)
0811 09:17:43.265678 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.265683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.266244 (+ 561us) server_negotiation.cc:407] Connection header received
0811 09:17:43.266478 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.266484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.266565 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.266656 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:43.267358 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.268286 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.270599 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.271342 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.271465 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.271532 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.271850 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.271957 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:17:43.273206 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.266355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:50857 (local address 127.17.16.65:45591)
0811 09:17:43.266564 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.266570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.266984 (+ 414us) server_negotiation.cc:407] Connection header received
0811 09:17:43.267185 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.267191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.267269 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.267348 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:43.267886 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.268692 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.270574 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.271371 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.271522 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.271594 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.272858 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.273000 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:17:43.273288 9079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.266233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45591 (local address 127.17.16.66:50857)
0811 09:17:43.266878 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:43.266897 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:43.267051 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:43.267377 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:43.267381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:43.267404 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:43.267743 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.267752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.268825 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.268829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.270422 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:43.270434 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.272540 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.272545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:43.272675 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.272706 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:17:43.272796 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 09:17:43.272943 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":69,"thread_start_us":314,"threads_started":1}
I20260811 09:17:43.273202 8968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9c55ebeaf941cd8cc5b55ea8ac0f92" candidate_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf2c040634154395bc16c3879da07551" is_pre_election: true
I20260811 09:17:43.273780 8968 raft_consensus.cc:2454] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be1b77a6ada0496f96ea1cbe1ef17f42 in term 0.
I20260811 09:17:43.274073 8578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9c55ebeaf941cd8cc5b55ea8ac0f92" candidate_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c007f901f0684f17a8b47286b2538651" is_pre_election: true
I20260811 09:17:43.274426 8674 leader_election.cc:302] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be1b77a6ada0496f96ea1cbe1ef17f42, bf2c040634154395bc16c3879da07551; no voters:
I20260811 09:17:43.274600 8578 raft_consensus.cc:2454] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be1b77a6ada0496f96ea1cbe1ef17f42 in term 0.
I20260811 09:17:43.274884 9076 raft_consensus.cc:2781] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:17:43.275008 9076 raft_consensus.cc:482] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:17:43.275120 9076 raft_consensus.cc:3037] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:43.277347 9076 raft_consensus.cc:504] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } }
I20260811 09:17:43.277889 9076 leader_election.cc:288] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [CANDIDATE]: Term 1 election: Requested vote from peers bf2c040634154395bc16c3879da07551 (127.17.16.67:42909), c007f901f0684f17a8b47286b2538651 (127.17.16.65:45591)
I20260811 09:17:43.278321 8968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9c55ebeaf941cd8cc5b55ea8ac0f92" candidate_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf2c040634154395bc16c3879da07551"
I20260811 09:17:43.278407 8578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9c55ebeaf941cd8cc5b55ea8ac0f92" candidate_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c007f901f0684f17a8b47286b2538651"
I20260811 09:17:43.278524 8968 raft_consensus.cc:3037] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:43.278621 8578 raft_consensus.cc:3037] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:43.281663 8968 raft_consensus.cc:2454] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be1b77a6ada0496f96ea1cbe1ef17f42 in term 1.
I20260811 09:17:43.281663 8578 raft_consensus.cc:2454] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be1b77a6ada0496f96ea1cbe1ef17f42 in term 1.
I20260811 09:17:43.282024 8674 leader_election.cc:302] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be1b77a6ada0496f96ea1cbe1ef17f42, c007f901f0684f17a8b47286b2538651; no voters:
I20260811 09:17:43.282307 9076 raft_consensus.cc:2781] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:17:43.282771 9076 raft_consensus.cc:686] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 1 LEADER]: Becoming Leader. State: Replica: be1b77a6ada0496f96ea1cbe1ef17f42, State: Running, Role: LEADER
I20260811 09:17:43.283214 9076 consensus_queue.cc:237] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } }
I20260811 09:17:43.287407 8347 catalog_manager.cc:5495] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 reported cstate change: term changed from 0 to 1, leader changed from <none> to be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66). New cstate: current_term: 1 leader_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260811 09:17:43.332870 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.327685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57374 (local address 127.17.16.66:35133)
0811 09:17:43.327876 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.327882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.327900 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:17:43.328097 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.328101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.328176 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.328302 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:17:43.328685 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.329439 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.330810 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.331224 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.331329 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.331835 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:17:43.332312 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:17:43.332440 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.332542 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.332619 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:17:43.340399 8578 raft_consensus.cc:1261] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Refusing update from remote peer be1b77a6ada0496f96ea1cbe1ef17f42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:17:43.340399 8968 raft_consensus.cc:1261] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551 [term 1 FOLLOWER]: Refusing update from remote peer be1b77a6ada0496f96ea1cbe1ef17f42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:17:43.341342 9080 consensus_queue.cc:1035] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:17:43.341713 9076 consensus_queue.cc:1035] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:17:43.367264 8348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:17:43.367543 8348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:17:43.370916 8348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Sending DeleteTablet for 3 replicas of tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92
I20260811 09:17:43.372057 8742 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:17:43 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:57372
I20260811 09:17:43.372095 8940 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:17:43 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:52588
I20260811 09:17:43.372095 8545 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:17:43 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:41628
I20260811 09:17:43.372330 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 8439
I20260811 09:17:43.386710 9102 tablet_replica.cc:330] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: stopping tablet replica
I20260811 09:17:43.387362 9100 tablet_replica.cc:330] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: stopping tablet replica
I20260811 09:17:43.387871 9100 raft_consensus.cc:2229] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:17:43.388574 9100 raft_consensus.cc:2258] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:17:43.389357 9098 tablet_replica.cc:330] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: stopping tablet replica
I20260811 09:17:43.389684 9098 raft_consensus.cc:2229] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:17:43.390092 9098 raft_consensus.cc:2258] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:17:43.390974 9102 raft_consensus.cc:2229] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:17:43.391451 9102 raft_consensus.cc:2258] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:17:43.391959 9098 ts_tablet_manager.cc:1905] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:17:43.394747 9102 ts_tablet_manager.cc:1905] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:17:43.396956 9098 ts_tablet_manager.cc:1918] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:17:43.397493 9098 log.cc:1196] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/4f9c55ebeaf941cd8cc5b55ea8ac0f92
I20260811 09:17:43.398049 9098 ts_tablet_manager.cc:1939] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P bf2c040634154395bc16c3879da07551: Deleting consensus metadata
I20260811 09:17:43.399870 8333 catalog_manager.cc:4841] TS bf2c040634154395bc16c3879da07551 (127.17.16.67:42909): tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92 (table test-table [id=3932e049f62d4bf48811756f427f4e68]) successfully deleted
Aug 11 09:17:43 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:17:43.401448 9100 ts_tablet_manager.cc:1905] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:17:43.401458 9102 ts_tablet_manager.cc:1918] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:17:43.401603 9102 log.cc:1196] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/4f9c55ebeaf941cd8cc5b55ea8ac0f92
I20260811 09:17:43.402215 9102 ts_tablet_manager.cc:1939] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P c007f901f0684f17a8b47286b2538651: Deleting consensus metadata
I20260811 09:17:43.403810 8333 catalog_manager.cc:4841] TS c007f901f0684f17a8b47286b2538651 (127.17.16.65:45591): tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92 (table test-table [id=3932e049f62d4bf48811756f427f4e68]) successfully deleted
I20260811 09:17:43.408867 9100 ts_tablet_manager.cc:1918] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:17:43.409081 9100 log.cc:1196] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/4f9c55ebeaf941cd8cc5b55ea8ac0f92
I20260811 09:17:43.409571 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.373622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.17.16.65:45591)
0811 09:17:43.373802 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.373808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.383550 (+ 9742us) server_negotiation.cc:407] Connection header received
0811 09:17:43.383999 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.384005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.384100 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.384172 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:43.386925 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.388081 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.391854 (+ 3773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.392493 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.392595 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.402157 (+ 9562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:43.402175 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:43.402211 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:43.402262 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:43.406020 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:43.406890 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:43.406899 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:43.406903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:43.407054 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:43.407517 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:43.407522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:43.407524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:43.407960 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:43.408242 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.408818 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.409201 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:17:43.409668 9100 ts_tablet_manager.cc:1939] T 4f9c55ebeaf941cd8cc5b55ea8ac0f92 P be1b77a6ada0496f96ea1cbe1ef17f42: Deleting consensus metadata
I20260811 09:17:43.411223 8629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:43.411283 8331 catalog_manager.cc:4841] TS be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133): tablet 4f9c55ebeaf941cd8cc5b55ea8ac0f92 (table test-table [id=3932e049f62d4bf48811756f427f4e68]) successfully deleted
I20260811 09:17:43.504259 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 8439
I20260811 09:17:43.534130 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 8643
Aug 11 09:17:43 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:17:43.550959 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.535013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57380 (local address 127.17.16.66:35133)
0811 09:17:43.535224 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.535229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.535250 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:17:43.535340 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.535345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.535418 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.535511 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:43.536823 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.537714 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.539529 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.539959 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.540058 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.545284 (+ 5226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:43.545296 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:43.545310 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:43.545345 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:43.548193 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:43.548838 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:43.548845 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:43.548847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:43.549032 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:43.549399 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:43.549405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:43.549408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:43.549766 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:43.550112 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.550369 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.550582 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:17:43.551772 8828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:43.664677 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 8643
I20260811 09:17:43.688799 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 8846
Aug 11 09:17:43 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:17:43.703758 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:43.689706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52596 (local address 127.17.16.67:42909)
0811 09:17:43.689878 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:43.689884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:43.689907 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:17:43.690106 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:43.690111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:43.690185 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:43.690272 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:43.691653 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.692364 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.693706 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:43.694100 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:43.694210 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:43.698401 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:43.698418 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:43.698435 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:43.698470 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:43.701049 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:43.701713 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:43.701722 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:43.701724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:43.701857 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:43.702228 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:43.702232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:43.702234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:43.702611 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:43.702906 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:43.703209 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 09:17:43.703435 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:17:43.704386 9031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:43.793059 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 8846
I20260811 09:17:43.814327 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 8302
I20260811 09:17:43.815594 8400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:43.925561 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 8302
I20260811 09:17:43.951493 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:37471
--webserver_interface=127.17.16.126
--webserver_port=46467
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:37471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:17:44.279074 9110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:44.279350 9110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:44.279479 9110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:44.286966 9110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:17:44.287067 9110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:44.287139 9110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:44.287226 9110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:17:44.287287 9110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:17:44.296833 9110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:37471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:37471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=46467
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:44.298197 9110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:44.299491 9110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:44.323558 9118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:44.323537 9122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:44.323503 9119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:44.324275 9110 server_base.cc:1034] running on GCE node
I20260811 09:17:44.325170 9110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:44.326694 9110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:44.327920 9110 hybrid_clock.cc:648] HybridClock initialized: now 1786439864327851 us; error 52 us; skew 500 ppm
I20260811 09:17:44.328405 9110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:44 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439864, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:44.332079 9110 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:17:44.333770 9110 webserver.cc:460] Webserver started at http://127.17.16.126:46467/ using document root <none> and password file <none>
I20260811 09:17:44.334529 9110 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:44.334682 9110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:44.340806 9110 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 09:17:44.343732 9133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:44.345070 9110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:17:44.345221 9110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "d1d4e62e0d3041138b484db176bb393c"
format_stamp: "Formatted at 2026-08-11 09:17:41 on dist-test-slave-ncp9"
server_key: "9773a101ac607923977c8e6a9ce12ee9"
server_key_iv: "562fc26efedcbc1a51eba2a56ed9ea53"
server_key_version: "encryptionkey@0"
I20260811 09:17:44.346240 9110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:44.364390 9110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:44.368286 9110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:44.368515 9110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:44.385913 9110 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:37471
I20260811 09:17:44.385946 9200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:37471 every 8 connection(s)
I20260811 09:17:44.387355 9110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:17:44.394153 9201 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:17:44.397467 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 9110
I20260811 09:17:44.397646 9201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Bootstrap starting.
I20260811 09:17:44.398480 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:45591
--local_ip_for_outbound_sockets=127.17.16.65
--tserver_master_addrs=127.17.16.126:37471
--webserver_port=43595
--webserver_interface=127.17.16.65
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:17:44.419876 9201 log.cc:824] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:44.432118 9201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:17:44.432622 9201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Bootstrap complete.
I20260811 09:17:44.441524 9201 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:44.442345 9201 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d4e62e0d3041138b484db176bb393c, State: Initialized, Role: FOLLOWER
I20260811 09:17:44.442982 9201 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:44.443207 9201 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:17:44.443322 9201 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:17:44.443483 9201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:17:44.447206 9201 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:44.447703 9201 leader_election.cc:302] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1d4e62e0d3041138b484db176bb393c; no voters:
I20260811 09:17:44.448292 9201 leader_election.cc:288] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:17:44.448431 9205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:17:44.449676 9205 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 2 LEADER]: Becoming Leader. State: Replica: d1d4e62e0d3041138b484db176bb393c, State: Running, Role: LEADER
I20260811 09:17:44.450228 9205 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:44.450624 9201 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:17:44.453141 9206 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d1d4e62e0d3041138b484db176bb393c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } } }
I20260811 09:17:44.453439 9206 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:44.452685 9208 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1d4e62e0d3041138b484db176bb393c. Latest consensus state: current_term: 2 leader_uuid: "d1d4e62e0d3041138b484db176bb393c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } } }
I20260811 09:17:44.453898 9208 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:44.457237 9220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:17:44.466531 9220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:17:44.467985 9220 catalog_manager.cc:1259] Loaded cluster ID: c39d92db02f14c958cfc1923f0df2bcf
I20260811 09:17:44.468088 9220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:17:44.471431 9220 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 09:17:44.490567 9220 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 09:17:44.518296 9220 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:17:44.520053 9220 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Loaded TSK: 0
I20260811 09:17:44.521615 9220 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:17:44.756206 9203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:44.756484 9203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:44.756610 9203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:44.763607 9203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:44.763716 9203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:44.763863 9203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:17:44.773213 9203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:45591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=43595
--tserver_master_addrs=127.17.16.126:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:44.774435 9203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:44.775779 9203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:44.789647 9246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:44.789775 9248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:44.790184 9251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:44.791322 9203 server_base.cc:1034] running on GCE node
I20260811 09:17:44.791929 9203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:44.792846 9203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:44.794092 9203 hybrid_clock.cc:648] HybridClock initialized: now 1786439864794049 us; error 36 us; skew 500 ppm
I20260811 09:17:44.794378 9203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:44 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439864, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:44.801386 9203 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:17:44.802858 9203 webserver.cc:460] Webserver started at http://127.17.16.65:43595/ using document root <none> and password file <none>
I20260811 09:17:44.803539 9203 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:44.803666 9203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:44.808857 9203 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:17:44.811461 9261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:44.812714 9203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:17:44.812861 9203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "c007f901f0684f17a8b47286b2538651"
format_stamp: "Formatted at 2026-08-11 09:17:41 on dist-test-slave-ncp9"
server_key: "16f777f49925bfe1b4d48d653e1e4a4d"
server_key_iv: "c103ff9c38c1538dabbfb2520f609141"
server_key_version: "encryptionkey@0"
I20260811 09:17:44.813807 9203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:44.836421 9203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:44.840076 9203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:44.840314 9203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:44.841269 9203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:44.842578 9203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:44.842665 9203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:44.842767 9203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:44.842840 9203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:44.872519 9203 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:45591
I20260811 09:17:44.872593 9435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:45591 every 8 connection(s)
I20260811 09:17:44.873940 9203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:17:44.878490 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 9203
I20260811 09:17:44.879077 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:35133
--local_ip_for_outbound_sockets=127.17.16.66
--tserver_master_addrs=127.17.16.126:37471
--webserver_port=46291
--webserver_interface=127.17.16.66
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:17:44 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439864, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:44.896030 9443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:44.876635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:49331 (local address 127.17.16.126:37471)
0811 09:17:44.877188 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:44.877200 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:44.877836 (+ 636us) server_negotiation.cc:407] Connection header received
0811 09:17:44.880012 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:44.880026 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:44.880331 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:44.880641 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:44.882164 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:44.883409 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:44.884942 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:44.885582 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:44.885756 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:44.889192 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:44.889203 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:44.889220 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:44.889262 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:44.891989 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:44.892816 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:44.892821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:44.892826 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:44.893011 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:44.893438 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:44.893441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:44.893443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:44.893866 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:44.894009 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:44.894781 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 09:17:44.895110 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":240,"threads_started":1}
I20260811 09:17:44.896378 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:44.876814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:49331)
0811 09:17:44.877672 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:44.877725 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:44.879495 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:44.880825 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:44.880838 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:44.881431 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:44.881947 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:44.881965 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:44.883575 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:44.883580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:44.884774 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:44.884798 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:44.885805 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:44.885812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:44.886058 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:44.886675 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:44.886696 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:44.888882 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:44.892134 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:44.892143 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:44.892158 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:44.892703 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:44.893118 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:44.893122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:44.893125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:44.893341 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:44.894007 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:44.894014 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:44.894356 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 09:17:44.894617 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 09:17:44.895095 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":328,"threads_started":1}
I20260811 09:17:44.899205 9440 heartbeater.cc:347] Connected to a master server at 127.17.16.126:37471
I20260811 09:17:44.899482 9440 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:44.900149 9440 heartbeater.cc:511] Master 127.17.16.126:37471 requested a full tablet report, sending...
I20260811 09:17:44.902488 9155 ts_manager.cc:194] Registered new tserver with Master: c007f901f0684f17a8b47286b2538651 (127.17.16.65:45591)
I20260811 09:17:44.904489 9155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:49331
W20260811 09:17:45.211472 9446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:45.211748 9446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:45.211863 9446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:45.219126 9446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:45.219229 9446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:45.219342 9446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:17:45.228533 9446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:35133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=46291
--tserver_master_addrs=127.17.16.126:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:45.229681 9446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:45.230882 9446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:45.245451 9456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:45.245404 9458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:45.245481 9461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:45.246255 9446 server_base.cc:1034] running on GCE node
I20260811 09:17:45.246846 9446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:45.247725 9446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:45.248945 9446 hybrid_clock.cc:648] HybridClock initialized: now 1786439865248888 us; error 42 us; skew 500 ppm
I20260811 09:17:45.249269 9446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:45 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439865, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:45.252426 9446 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:17:45.253818 9446 webserver.cc:460] Webserver started at http://127.17.16.66:46291/ using document root <none> and password file <none>
I20260811 09:17:45.254453 9446 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:45.254590 9446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:45.259636 9446 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 09:17:45.262152 9474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:45.263357 9446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 09:17:45.263507 9446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "be1b77a6ada0496f96ea1cbe1ef17f42"
format_stamp: "Formatted at 2026-08-11 09:17:42 on dist-test-slave-ncp9"
server_key: "2356194807b09d431df3655bf32de647"
server_key_iv: "d07fda8a4c287fa5f360bd62bbc1f0d9"
server_key_version: "encryptionkey@0"
I20260811 09:17:45.264423 9446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:45.289809 9446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:45.293404 9446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:45.293651 9446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:45.294590 9446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:45.295876 9446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:45.295976 9446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:45.296084 9446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:45.296154 9446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:45.325819 9446 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:35133
I20260811 09:17:45.325886 9673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:35133 every 8 connection(s)
I20260811 09:17:45.327123 9446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:17:45.328217 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 9446
I20260811 09:17:45.329000 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:42909
--local_ip_for_outbound_sockets=127.17.16.67
--tserver_master_addrs=127.17.16.126:37471
--webserver_port=40235
--webserver_interface=127.17.16.67
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:17:45 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439865, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:45.346223 9443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.329760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:52351 (local address 127.17.16.126:37471)
0811 09:17:45.329968 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:45.329974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:45.331144 (+ 1170us) server_negotiation.cc:407] Connection header received
0811 09:17:45.332950 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:45.332956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:45.333037 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:45.333144 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:45.334488 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.335442 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.336957 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.337512 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.337629 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.340834 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:45.340839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:45.340842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:45.340875 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:45.343376 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:45.344036 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:45.344040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:45.344042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:45.344171 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:45.344573 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:45.344576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:45.344578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:45.344862 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:45.345003 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:45.345796 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 09:17:45.345975 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:17:45.347429 9679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.330202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:52351)
0811 09:17:45.330973 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:45.331016 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:45.332675 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:45.333316 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:45.333324 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:45.333794 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:45.334313 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:45.334329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.335603 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.335606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:45.336790 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:45.336803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.337698 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.337703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:45.337884 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.338464 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:45.338484 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:45.340638 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:45.343508 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:45.343516 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:45.343530 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:45.343922 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:45.344284 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:45.344290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:45.344294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:45.344466 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:45.345038 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:45.345047 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:45.345415 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 09:17:45.345721 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 09:17:45.346196 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":322,"threads_started":1}
I20260811 09:17:45.348793 9676 heartbeater.cc:347] Connected to a master server at 127.17.16.126:37471
I20260811 09:17:45.349108 9676 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:45.349852 9676 heartbeater.cc:511] Master 127.17.16.126:37471 requested a full tablet report, sending...
I20260811 09:17:45.351280 9155 ts_manager.cc:194] Registered new tserver with Master: be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133)
I20260811 09:17:45.352159 9155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:52351
W20260811 09:17:45.656790 9680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:45.657083 9680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:45.657224 9680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:45.664212 9680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:45.664321 9680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:45.664479 9680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:17:45.673928 9680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:42909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=40235
--tserver_master_addrs=127.17.16.126:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:45.675107 9680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:45.676337 9680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:45.688645 9688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:45.688621 9689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:45.688582 9691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:45.689208 9680 server_base.cc:1034] running on GCE node
I20260811 09:17:45.689783 9680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:45.690589 9680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:45.691779 9680 hybrid_clock.cc:648] HybridClock initialized: now 1786439865691730 us; error 38 us; skew 500 ppm
I20260811 09:17:45.692059 9680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:45 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439865, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:45.694907 9680 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:17:45.696331 9680 webserver.cc:460] Webserver started at http://127.17.16.67:40235/ using document root <none> and password file <none>
I20260811 09:17:45.696987 9680 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:45.697124 9680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:45.702059 9680 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260811 09:17:45.704535 9699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:45.705755 9680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 09:17:45.705904 9680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "bf2c040634154395bc16c3879da07551"
format_stamp: "Formatted at 2026-08-11 09:17:42 on dist-test-slave-ncp9"
server_key: "cf2a47a5d6657ae872ae22931f59c31e"
server_key_iv: "a939ba571e330ced749dd005a0fc3a22"
server_key_version: "encryptionkey@0"
I20260811 09:17:45.706862 9680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:45.722479 9680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:45.726042 9680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:45.726284 9680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:45.727175 9680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:45.728420 9680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:45.728513 9680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:45.728618 9680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:45.728691 9680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:45.758379 9680 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:42909
I20260811 09:17:45.758422 9874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:42909 every 8 connection(s)
I20260811 09:17:45.759748 9680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:17:45.766621 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 9680
Aug 11 09:17:45 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439865, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:17:45.780382 9443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.762128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:58139 (local address 127.17.16.126:37471)
0811 09:17:45.762304 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:45.762308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:45.763387 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 09:17:45.765134 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:45.765138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:45.765207 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:45.765282 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:45.766846 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.767896 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.770205 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.770834 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.770952 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.774734 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:45.774739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:45.774743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:45.774773 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:45.777504 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:45.778186 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:45.778190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:45.778193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:45.778316 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:45.778745 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:45.778749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:45.778751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:45.779040 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:45.779202 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:45.779842 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 09:17:45.780061 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":59}
I20260811 09:17:45.781402 9877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.762564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:58139)
0811 09:17:45.763227 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:45.763263 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:45.764842 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:45.765462 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:45.765470 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:45.765949 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:45.766666 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:45.766693 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.768155 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.768161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:45.770065 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:45.770080 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.771039 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.771044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:45.771230 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.771885 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:45.771914 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:45.774535 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:45.777637 (+ 3102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:45.777645 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:45.777660 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:45.778063 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:45.778419 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:45.778424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:45.778428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:45.778625 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:45.779163 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:45.779172 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:45.779493 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 09:17:45.779714 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 09:17:45.780184 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":247,"threads_started":1}
I20260811 09:17:45.781942 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.767567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.17.16.126:37471)
0811 09:17:45.768326 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:45.768330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:45.768345 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:17:45.768384 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:45.768387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:45.768449 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:45.768529 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:45.770467 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.771424 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.773270 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.773935 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.774053 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.776718 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:45.776723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:45.776726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:45.776764 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:45.779308 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:45.779984 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:45.779988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:45.779991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:45.780105 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:45.780498 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:45.780503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:45.780505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:45.780792 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:45.781042 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:45.781354 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 09:17:45.781551 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":675,"thread_start_us":560,"threads_started":1}
I20260811 09:17:45.782755 9875 heartbeater.cc:347] Connected to a master server at 127.17.16.126:37471
I20260811 09:17:45.783077 9875 heartbeater.cc:464] Registering TS with master...
I20260811 09:17:45.783823 9875 heartbeater.cc:511] Master 127.17.16.126:37471 requested a full tablet report, sending...
I20260811 09:17:45.785233 9155 ts_manager.cc:194] Registered new tserver with Master: bf2c040634154395bc16c3879da07551 (127.17.16.67:42909)
I20260811 09:17:45.785990 9155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:58139
I20260811 09:17:45.796408 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:17:45.802790 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.797357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45950 (local address 127.17.16.126:37471)
0811 09:17:45.797579 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:45.797585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:45.797858 (+ 273us) server_negotiation.cc:407] Connection header received
0811 09:17:45.798035 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:45.798039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:45.798170 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:45.798286 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:17:45.798632 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.799293 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.800748 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.801207 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.801299 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.801748 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:17:45.802289 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:17:45.802420 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:45.802465 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 09:17:45.802528 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:17:45.803720 9155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:17:45.810943 9155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:17:45.832690 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.823427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57384 (local address 127.17.16.66:35133)
0811 09:17:45.825408 (+ 1981us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:45.825416 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:45.825451 (+ 35us) server_negotiation.cc:407] Connection header received
0811 09:17:45.825548 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:45.825555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:45.825770 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:45.826672 (+ 902us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:45.826976 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.828336 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.830608 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.831528 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.831639 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.831876 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:45.832165 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 09:17:45.832333 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1824,"thread_start_us":1688,"threads_started":1}
I20260811 09:17:45.832784 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.822828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35133 (local address 127.0.0.1:57384)
0811 09:17:45.823883 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:45.823900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:45.824005 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:45.825992 (+ 1987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:45.825997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:45.826038 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:45.826469 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:45.826480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.828509 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.828514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:45.830458 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:45.830470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.831735 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.831741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:45.831951 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.831988 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 09:17:45.832100 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 09:17:45.832322 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":25,"thread_start_us":4092,"threads_started":1}
I20260811 09:17:45.832841 9888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.822208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45591 (local address 127.0.0.1:41640)
0811 09:17:45.823103 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:45.823143 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:45.823308 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:45.825548 (+ 2240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:45.825556 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:45.825588 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:45.825980 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:45.825996 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.827781 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.827785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:45.829875 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:45.829891 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.831954 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.831961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:45.832141 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.832205 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 09:17:45.832275 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 09:17:45.832397 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":3,"thread_start_us":451,"threads_started":1}
I20260811 09:17:45.834338 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.822434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.17.16.65:45591)
0811 09:17:45.825027 (+ 2593us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:45.825033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:45.825056 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:17:45.825132 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:45.825145 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:45.825319 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:45.825486 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:45.826215 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.827665 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.830062 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.831670 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.833606 (+ 1936us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.833789 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:45.833857 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 09:17:45.834015 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2440,"thread_start_us":2304,"threads_started":1}
I20260811 09:17:45.836261 9561 tablet_service.cc:1467] Processing CreateTablet for tablet fa1610b79ebe4dd3ac8aac37100dc14d (DEFAULT_TABLE table=test-table [id=689a5eb4112e495480aff22a5502e7e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:17:45.837673 9561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa1610b79ebe4dd3ac8aac37100dc14d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:45.838770 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.823412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:42909 (local address 127.0.0.1:52598)
0811 09:17:45.827159 (+ 3747us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:45.827177 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:45.827294 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:45.828071 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:45.828075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:45.828098 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:45.828682 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:45.828696 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.830663 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.830668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:45.832745 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:45.832761 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.837995 (+ 5234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.838001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:45.838175 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.838207 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:17:45.838342 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 09:17:45.838469 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3626,"thread_start_us":877,"threads_started":1}
I20260811 09:17:45.839278 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:45.823993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.17.16.67:42909)
0811 09:17:45.824751 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:45.824760 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:45.827285 (+ 2525us) server_negotiation.cc:407] Connection header received
0811 09:17:45.827426 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:45.827434 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:45.827632 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:45.827812 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:45.828949 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.830475 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.832973 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:45.837492 (+ 4519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:45.837590 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:45.837797 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:45.838410 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 09:17:45.838581 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":422,"threads_started":1}
I20260811 09:17:45.840030 9336 tablet_service.cc:1467] Processing CreateTablet for tablet fa1610b79ebe4dd3ac8aac37100dc14d (DEFAULT_TABLE table=test-table [id=689a5eb4112e495480aff22a5502e7e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:17:45.841496 9336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa1610b79ebe4dd3ac8aac37100dc14d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:45.842666 9774 tablet_service.cc:1467] Processing CreateTablet for tablet fa1610b79ebe4dd3ac8aac37100dc14d (DEFAULT_TABLE table=test-table [id=689a5eb4112e495480aff22a5502e7e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:17:45.843959 9774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa1610b79ebe4dd3ac8aac37100dc14d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:17:45.852876 9898 tablet_bootstrap.cc:492] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Bootstrap starting.
I20260811 09:17:45.854715 9899 tablet_bootstrap.cc:492] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: Bootstrap starting.
I20260811 09:17:45.857648 9898 tablet_bootstrap.cc:654] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:45.857694 9899 tablet_bootstrap.cc:654] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:45.858165 9902 tablet_bootstrap.cc:492] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: Bootstrap starting.
I20260811 09:17:45.858929 9899 log.cc:824] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:45.858929 9898 log.cc:824] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:45.861320 9902 tablet_bootstrap.cc:654] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: Neither blocks nor log segments found. Creating new log.
I20260811 09:17:45.861855 9899 tablet_bootstrap.cc:492] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: No bootstrap required, opened a new log
I20260811 09:17:45.862144 9899 ts_tablet_manager.cc:1397] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:17:45.862356 9902 log.cc:824] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:45.863751 9898 tablet_bootstrap.cc:492] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: No bootstrap required, opened a new log
I20260811 09:17:45.863996 9898 ts_tablet_manager.cc:1397] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 09:17:45.864753 9902 tablet_bootstrap.cc:492] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: No bootstrap required, opened a new log
I20260811 09:17:45.865005 9902 ts_tablet_manager.cc:1397] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260811 09:17:45.871879 9902 raft_consensus.cc:348] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:45.872325 9902 raft_consensus.cc:374] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:45.872418 9902 raft_consensus.cc:729] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf2c040634154395bc16c3879da07551, State: Initialized, Role: FOLLOWER
I20260811 09:17:45.872234 9899 raft_consensus.cc:348] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:45.872625 9899 raft_consensus.cc:374] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:45.872706 9899 raft_consensus.cc:729] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be1b77a6ada0496f96ea1cbe1ef17f42, State: Initialized, Role: FOLLOWER
I20260811 09:17:45.873380 9902 consensus_queue.cc:260] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:45.873407 9899 consensus_queue.cc:260] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:45.875185 9676 heartbeater.cc:503] Master 127.17.16.126:37471 was elected leader, sending a full tablet report...
I20260811 09:17:45.875391 9902 ts_tablet_manager.cc:1428] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 09:17:45.875738 9899 ts_tablet_manager.cc:1428] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 09:17:45.876603 9875 heartbeater.cc:503] Master 127.17.16.126:37471 was elected leader, sending a full tablet report...
I20260811 09:17:45.879438 9898 raft_consensus.cc:348] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:45.879865 9898 raft_consensus.cc:374] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:17:45.879972 9898 raft_consensus.cc:729] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c007f901f0684f17a8b47286b2538651, State: Initialized, Role: FOLLOWER
I20260811 09:17:45.880599 9898 consensus_queue.cc:260] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:45.881824 9440 heartbeater.cc:503] Master 127.17.16.126:37471 was elected leader, sending a full tablet report...
I20260811 09:17:45.882419 9898 ts_tablet_manager.cc:1428] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Time spent starting tablet: real 0.018s user 0.013s sys 0.000s
W20260811 09:17:46.011299 9876 tablet.cc:2369] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:17:46.053563 9909 raft_consensus.cc:482] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:17:46.053854 9909 raft_consensus.cc:504] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:46.055518 9909 leader_election.cc:288] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf2c040634154395bc16c3879da07551 (127.17.16.67:42909), be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133)
I20260811 09:17:46.062168 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.055948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:56905 (local address 127.17.16.66:35133)
0811 09:17:46.056153 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:46.056159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:46.056848 (+ 689us) server_negotiation.cc:407] Connection header received
0811 09:17:46.057099 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:46.057103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:46.057167 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:46.057252 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:46.057893 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.058738 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.060492 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.061308 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.061450 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:46.061524 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:46.061800 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 09:17:46.061925 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:17:46.062309 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.055998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35133 (local address 127.17.16.65:56905)
0811 09:17:46.056728 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:46.056742 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:46.056935 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:46.057292 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:46.057297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:46.057320 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:46.057721 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:46.057737 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.058873 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.058878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:46.060327 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:46.060341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.061507 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.061514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:46.061654 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:46.061679 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:17:46.061764 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 09:17:46.061889 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":1,"thread_start_us":396,"threads_started":1}
I20260811 09:17:46.062633 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.055998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:42909 (local address 127.17.16.65:34705)
0811 09:17:46.057120 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:46.057134 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:46.057267 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:46.057585 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:46.057589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:46.057618 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:17:46.058014 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:46.058034 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.059267 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.059273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:46.060651 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:46.060664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.061836 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.061844 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:46.061980 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:46.062003 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:17:46.062094 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 09:17:46.062229 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":28,"spinlock_wait_cycles":24448,"thread_start_us":779,"threads_started":1}
I20260811 09:17:46.062825 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.056072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:34705 (local address 127.17.16.67:42909)
0811 09:17:46.056251 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:46.056255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:46.057240 (+ 985us) server_negotiation.cc:407] Connection header received
0811 09:17:46.057396 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:46.057403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:46.057465 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:46.057553 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:17:46.058197 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.059136 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.060801 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.061626 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.061766 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:46.061835 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:46.062138 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 09:17:46.062596 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:17:46.063218 9596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1610b79ebe4dd3ac8aac37100dc14d" candidate_uuid: "c007f901f0684f17a8b47286b2538651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" is_pre_election: true
I20260811 09:17:46.063681 9596 raft_consensus.cc:2454] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c007f901f0684f17a8b47286b2538651 in term 0.
I20260811 09:17:46.063649 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1610b79ebe4dd3ac8aac37100dc14d" candidate_uuid: "c007f901f0684f17a8b47286b2538651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf2c040634154395bc16c3879da07551" is_pre_election: true
I20260811 09:17:46.064107 9801 raft_consensus.cc:2454] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c007f901f0684f17a8b47286b2538651 in term 0.
I20260811 09:17:46.064308 9265 leader_election.cc:302] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be1b77a6ada0496f96ea1cbe1ef17f42, c007f901f0684f17a8b47286b2538651; no voters:
I20260811 09:17:46.064648 9909 raft_consensus.cc:2781] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:17:46.064775 9909 raft_consensus.cc:482] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:17:46.064867 9909 raft_consensus.cc:3037] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:46.067009 9909 raft_consensus.cc:504] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:46.067613 9909 leader_election.cc:288] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 1 election: Requested vote from peers bf2c040634154395bc16c3879da07551 (127.17.16.67:42909), be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133)
I20260811 09:17:46.067999 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1610b79ebe4dd3ac8aac37100dc14d" candidate_uuid: "c007f901f0684f17a8b47286b2538651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf2c040634154395bc16c3879da07551"
I20260811 09:17:46.068008 9596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1610b79ebe4dd3ac8aac37100dc14d" candidate_uuid: "c007f901f0684f17a8b47286b2538651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42"
I20260811 09:17:46.068154 9801 raft_consensus.cc:3037] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:46.068176 9596 raft_consensus.cc:3037] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:17:46.070384 9801 raft_consensus.cc:2454] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c007f901f0684f17a8b47286b2538651 in term 1.
I20260811 09:17:46.070392 9596 raft_consensus.cc:2454] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c007f901f0684f17a8b47286b2538651 in term 1.
I20260811 09:17:46.070736 9265 leader_election.cc:302] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: be1b77a6ada0496f96ea1cbe1ef17f42, bf2c040634154395bc16c3879da07551, c007f901f0684f17a8b47286b2538651; no voters:
I20260811 09:17:46.071033 9909 raft_consensus.cc:2781] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:17:46.071425 9909 raft_consensus.cc:686] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 LEADER]: Becoming Leader. State: Replica: c007f901f0684f17a8b47286b2538651, State: Running, Role: LEADER
I20260811 09:17:46.071866 9909 consensus_queue.cc:237] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:46.076308 9155 catalog_manager.cc:5495] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 reported cstate change: term changed from 0 to 1, leader changed from <none> to c007f901f0684f17a8b47286b2538651 (127.17.16.65). New cstate: current_term: 1 leader_uuid: "c007f901f0684f17a8b47286b2538651" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } health_report { overall_health: UNKNOWN } } }
W20260811 09:17:46.078965 9677 tablet.cc:2369] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:17:46.126161 9441 tablet.cc:2369] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:17:46.127902 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.122588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41654 (local address 127.17.16.65:45591)
0811 09:17:46.122767 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:46.122771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:46.122884 (+ 113us) server_negotiation.cc:407] Connection header received
0811 09:17:46.123070 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:46.123074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:46.123166 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:46.123281 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:17:46.123574 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.124298 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.125796 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.126433 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.126559 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:46.126965 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:17:46.127446 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:17:46.127551 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:46.127629 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 09:17:46.127699 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:17:46.135526 9801 raft_consensus.cc:1261] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551 [term 1 FOLLOWER]: Refusing update from remote peer c007f901f0684f17a8b47286b2538651: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:17:46.135733 9596 raft_consensus.cc:1261] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42 [term 1 FOLLOWER]: Refusing update from remote peer c007f901f0684f17a8b47286b2538651: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:17:46.136480 9909 consensus_queue.cc:1035] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:17:46.136773 9913 consensus_queue.cc:1035] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:17:46.148144 9917 mvcc.cc:205] Tried to move back new op lower bound from 7317257691682033664 to 7317257691433693184. Current Snapshot: MvccSnapshot[applied={T|T < 7317257691682033664}]
I20260811 09:17:46.148343 9918 mvcc.cc:205] Tried to move back new op lower bound from 7317257691682033664 to 7317257691433693184. Current Snapshot: MvccSnapshot[applied={T|T < 7317257691682033664}]
I20260811 09:17:46.161330 9155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:17:46.161603 9155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:17:46.165153 9155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Sending DeleteTablet for 3 replicas of tablet fa1610b79ebe4dd3ac8aac37100dc14d
I20260811 09:17:46.166251 9336 tablet_service.cc:1514] Processing DeleteTablet for tablet fa1610b79ebe4dd3ac8aac37100dc14d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:17:46 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:41640
I20260811 09:17:46.166297 9774 tablet_service.cc:1514] Processing DeleteTablet for tablet fa1610b79ebe4dd3ac8aac37100dc14d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:17:46 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:52598
I20260811 09:17:46.166343 9561 tablet_service.cc:1514] Processing DeleteTablet for tablet fa1610b79ebe4dd3ac8aac37100dc14d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:17:46 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:57384
I20260811 09:17:46.169549 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 9203
I20260811 09:17:46.180015 9937 tablet_replica.cc:330] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: stopping tablet replica
I20260811 09:17:46.180475 9937 raft_consensus.cc:2229] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:17:46.181003 9937 raft_consensus.cc:2258] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:17:46.190129 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.171032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41670 (local address 127.17.16.65:45591)
0811 09:17:46.171220 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:46.171226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:46.171244 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:17:46.171312 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:46.171318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:46.171396 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:46.171499 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:46.173600 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.174966 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.177012 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.177697 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.177818 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:46.182214 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:46.182229 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:46.182247 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:46.182291 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:46.186914 (+ 4623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:46.187801 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:46.187811 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:46.187816 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:46.187999 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:46.188396 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:46.188402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:46.188405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:46.188869 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:46.189250 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:46.189527 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 09:17:46.189752 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:17:46.190795 9429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:46.192500 9935 tablet_replica.cc:330] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: stopping tablet replica
I20260811 09:17:46.193042 9935 raft_consensus.cc:2229] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:17:46.193593 9935 raft_consensus.cc:2258] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:17:46.193840 9937 ts_tablet_manager.cc:1905] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:17:46.195741 9935 ts_tablet_manager.cc:1905] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:17:46.202562 9937 ts_tablet_manager.cc:1918] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:17:46.202713 9937 log.cc:1196] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/fa1610b79ebe4dd3ac8aac37100dc14d
I20260811 09:17:46.202790 9935 ts_tablet_manager.cc:1918] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:17:46.202953 9935 log.cc:1196] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/fa1610b79ebe4dd3ac8aac37100dc14d
I20260811 09:17:46.203459 9935 ts_tablet_manager.cc:1939] T fa1610b79ebe4dd3ac8aac37100dc14d P bf2c040634154395bc16c3879da07551: Deleting consensus metadata
I20260811 09:17:46.203455 9937 ts_tablet_manager.cc:1939] T fa1610b79ebe4dd3ac8aac37100dc14d P be1b77a6ada0496f96ea1cbe1ef17f42: Deleting consensus metadata
I20260811 09:17:46.205045 9138 catalog_manager.cc:4841] TS bf2c040634154395bc16c3879da07551 (127.17.16.67:42909): tablet fa1610b79ebe4dd3ac8aac37100dc14d (table test-table [id=689a5eb4112e495480aff22a5502e7e0]) successfully deleted
I20260811 09:17:46.205073 9137 catalog_manager.cc:4841] TS be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133): tablet fa1610b79ebe4dd3ac8aac37100dc14d (table test-table [id=689a5eb4112e495480aff22a5502e7e0]) successfully deleted
I20260811 09:17:46.295439 9936 tablet_replica.cc:330] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: stopping tablet replica
I20260811 09:17:46.296020 9936 raft_consensus.cc:2229] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:17:46.296763 9936 raft_consensus.cc:2258] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:17:46.297652 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 9203
W20260811 09:17:46.322711 9138 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.16.65:45591 (error 108)
I20260811 09:17:46.322942 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 9446
I20260811 09:17:46.324404 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.323946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45591 (local address 127.0.0.1:41672)
0811 09:17:46.324118 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:46.324228 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 09:17:46.324659 9138 catalog_manager.cc:4584] TS c007f901f0684f17a8b47286b2538651 (127.17.16.65:45591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa1610b79ebe4dd3ac8aac37100dc14d: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45591: connect: Connection refused (error 111)
I20260811 09:17:46.337932 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.323666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.17.16.66:35133)
0811 09:17:46.323832 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:46.323838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:46.323858 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:17:46.324005 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:46.324010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:46.324095 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:46.324195 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:46.325767 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.326912 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.328185 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.328863 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.329032 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:46.331616 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:46.331627 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:46.331650 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:46.331706 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:46.335264 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:46.335961 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:46.335970 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:46.335974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:46.336114 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:46.336447 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:46.336451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:46.336455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:46.336880 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:46.337179 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:46.337367 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 09:17:46.337573 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 09:17:46.338603 9663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:46.378225 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.377791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45591 (local address 127.0.0.1:41688)
0811 09:17:46.377984 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:46.378077 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 09:17:46.379155 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.378849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45591 (local address 127.0.0.1:41704)
0811 09:17:46.378964 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:46.379035 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 09:17:46.426342 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 9446
I20260811 09:17:46.439462 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.438991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45591 (local address 127.0.0.1:41714)
0811 09:17:46.439201 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:46.439289 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 09:17:46.440440 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.440150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45591 (local address 127.0.0.1:41726)
0811 09:17:46.440266 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:46.440337 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 09:17:46.450700 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 9680
I20260811 09:17:46.463035 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.451420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52604 (local address 127.17.16.67:42909)
0811 09:17:46.451569 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:46.451576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:46.451637 (+ 61us) server_negotiation.cc:407] Connection header received
0811 09:17:46.451816 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:46.451820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:46.451888 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:46.451987 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:46.453375 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.454199 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.455375 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:46.455743 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:46.455826 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:46.458253 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:46.458266 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:46.458283 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:46.458312 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:46.460795 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:46.461431 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:46.461438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:46.461441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:46.461570 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:46.461850 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:46.461853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:46.461854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:46.462173 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:46.462405 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:46.462624 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 09:17:46.462780 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:17:46.463892 9868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:46.521214 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.520679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45591 (local address 127.0.0.1:41732)
0811 09:17:46.520911 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:46.521017 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 09:17:46.522591 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:46.522174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:45591 (local address 127.0.0.1:41748)
0811 09:17:46.522403 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:46.522481 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 09:17:46.550979 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 9680
I20260811 09:17:46.573029 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 9110
I20260811 09:17:46.574163 9196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:17:46.686777 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 9110
I20260811 09:17:46.716146 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:37471
--webserver_interface=127.17.16.126
--webserver_port=46467
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:37471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:17:47.043037 9947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:47.043311 9947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:47.043452 9947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:47.050376 9947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:17:47.050489 9947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:47.050570 9947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:47.050642 9947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:17:47.050704 9947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:17:47.060243 9947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:37471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:37471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=46467
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:47.061578 9947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:47.062889 9947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:47.073812 9953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:47.074189 9958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:47.074333 9956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:47.075103 9947 server_base.cc:1034] running on GCE node
I20260811 09:17:47.075872 9947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:47.077248 9947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:47.078496 9947 hybrid_clock.cc:648] HybridClock initialized: now 1786439867078410 us; error 49 us; skew 500 ppm
I20260811 09:17:47.078837 9947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:47 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439867, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:47.081836 9947 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:17:47.083278 9947 webserver.cc:460] Webserver started at http://127.17.16.126:46467/ using document root <none> and password file <none>
I20260811 09:17:47.083902 9947 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:47.084005 9947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:47.089371 9947 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:17:47.092047 9966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:47.093375 9947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:17:47.093513 9947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "d1d4e62e0d3041138b484db176bb393c"
format_stamp: "Formatted at 2026-08-11 09:17:41 on dist-test-slave-ncp9"
server_key: "9773a101ac607923977c8e6a9ce12ee9"
server_key_iv: "562fc26efedcbc1a51eba2a56ed9ea53"
server_key_version: "encryptionkey@0"
I20260811 09:17:47.094430 9947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:47.114078 9947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:47.117766 9947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:47.117982 9947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:47.135344 9947 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:37471
I20260811 09:17:47.135351 10050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:37471 every 8 connection(s)
I20260811 09:17:47.136771 9947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:17:47.140832 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 9947
I20260811 09:17:47.141630 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:45591
--local_ip_for_outbound_sockets=127.17.16.65
--tserver_master_addrs=127.17.16.126:37471
--webserver_port=43595
--webserver_interface=127.17.16.65
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:17:47.143457 10052 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:17:47.146577 10052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Bootstrap starting.
I20260811 09:17:47.167518 10052 log.cc:824] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:47.183893 10052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:17:47.184406 10052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c: Bootstrap complete.
I20260811 09:17:47.193285 10052 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:47.194151 10052 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d4e62e0d3041138b484db176bb393c, State: Initialized, Role: FOLLOWER
I20260811 09:17:47.194804 10052 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:47.195026 10052 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:17:47.195137 10052 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:17:47.195293 10052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 2 FOLLOWER]: Advancing to term 3
I20260811 09:17:47.199302 10052 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:47.199785 10052 leader_election.cc:302] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1d4e62e0d3041138b484db176bb393c; no voters:
I20260811 09:17:47.200376 10052 leader_election.cc:288] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 09:17:47.200562 10063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 3 FOLLOWER]: Leader election won for term 3
I20260811 09:17:47.202208 10052 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:17:47.205564 10063 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [term 3 LEADER]: Becoming Leader. State: Replica: d1d4e62e0d3041138b484db176bb393c, State: Running, Role: LEADER
I20260811 09:17:47.206139 10063 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } }
I20260811 09:17:47.208868 10064 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d1d4e62e0d3041138b484db176bb393c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } } }
I20260811 09:17:47.208886 10065 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1d4e62e0d3041138b484db176bb393c. Latest consensus state: current_term: 3 leader_uuid: "d1d4e62e0d3041138b484db176bb393c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1d4e62e0d3041138b484db176bb393c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37471 } } }
I20260811 09:17:47.209260 10065 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:47.209265 10064 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1d4e62e0d3041138b484db176bb393c [sys.catalog]: This master's current role is: LEADER
I20260811 09:17:47.218044 10073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:17:47.226570 10073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:17:47.227989 10073 catalog_manager.cc:1259] Loaded cluster ID: c39d92db02f14c958cfc1923f0df2bcf
I20260811 09:17:47.228088 10073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 09:17:47.230172 10073 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f39adf549ad google::LogMessage::Fail() at ??:0
@ 0x7f39adf5788a google::LogMessage::SendToLog() at ??:0
@ 0x7f39adf544de google::LogMessage::Flush() at ??:0
@ 0x7f39adf55a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f39c3987512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f39c3984976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f39c3a0c610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f39c3a0c439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f39b0ba0345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f39b0ba391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f39b0ba3799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f39b0b7eeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f39b3e586db start_thread at ??:0
@ 0x7f39acdee61f clone at ??:0
W20260811 09:17:47.559780 10055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:47.560039 10055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:47.560132 10055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:47.567207 10055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:47.567305 10055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:47.567420 10055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:17:47.576656 10055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:45591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=43595
--tserver_master_addrs=127.17.16.126:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:47.577848 10055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:47.579104 10055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:47.591634 10126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:47.591570 10127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:47.591591 10130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:47.592658 10055 server_base.cc:1034] running on GCE node
I20260811 09:17:47.593295 10055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:47.594132 10055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:47.595314 10055 hybrid_clock.cc:648] HybridClock initialized: now 1786439867595265 us; error 46 us; skew 500 ppm
I20260811 09:17:47.595629 10055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:47 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439867, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:47.606626 10055 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:17:47.608018 10055 webserver.cc:460] Webserver started at http://127.17.16.65:43595/ using document root <none> and password file <none>
I20260811 09:17:47.608659 10055 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:47.608791 10055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:47.613957 10055 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:17:47.619124 10143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:47.620388 10055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:17:47.620572 10055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "c007f901f0684f17a8b47286b2538651"
format_stamp: "Formatted at 2026-08-11 09:17:41 on dist-test-slave-ncp9"
server_key: "16f777f49925bfe1b4d48d653e1e4a4d"
server_key_iv: "c103ff9c38c1538dabbfb2520f609141"
server_key_version: "encryptionkey@0"
I20260811 09:17:47.621555 10055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:47.645385 10055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:47.650147 10055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:47.650452 10055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:47.651597 10055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:47.654788 10154 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 09:17:47.662814 10055 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 09:17:47.662945 10055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.002s
I20260811 09:17:47.663069 10055 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 09:17:47.667136 10055 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 09:17:47.667229 10055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.001s sys 0.001s
I20260811 09:17:47.667660 10154 tablet_bootstrap.cc:492] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Bootstrap starting.
I20260811 09:17:47.677416 10154 log.cc:824] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Log is configured to *not* fsync() on all Append() calls
I20260811 09:17:47.692996 10154 tablet_bootstrap.cc:492] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:17:47.693953 10154 tablet_bootstrap.cc:492] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Bootstrap complete.
I20260811 09:17:47.694571 10154 ts_tablet_manager.cc:1397] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Time spent bootstrapping tablet: real 0.027s user 0.022s sys 0.004s
I20260811 09:17:47.707541 10055 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:45591
I20260811 09:17:47.707602 10324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:45591 every 8 connection(s)
I20260811 09:17:47.706791 10154 raft_consensus.cc:348] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:47.708241 10154 raft_consensus.cc:729] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c007f901f0684f17a8b47286b2538651, State: Initialized, Role: FOLLOWER
I20260811 09:17:47.709067 10055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:17:47.708873 10154 consensus_queue.cc:260] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:47.710863 10154 ts_tablet_manager.cc:1428] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651: Time spent starting tablet: real 0.016s user 0.008s sys 0.008s
I20260811 09:17:47.711292 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 10055
I20260811 09:17:47.712152 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:35133
--local_ip_for_outbound_sockets=127.17.16.66
--tserver_master_addrs=127.17.16.126:37471
--webserver_port=46291
--webserver_interface=127.17.16.66
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:17:47.718954 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:47.712055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:58571)
0811 09:17:47.717679 (+ 5624us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:47.717965 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5334,"thread_start_us":5128,"threads_started":1}
W20260811 09:17:47.719497 10328 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:37471 (0 consecutive failures): Network error: Failed to ping master at 127.17.16.126:37471: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
I20260811 09:17:47.720350 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:47.720041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:38419)
0811 09:17:47.720180 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:47.720247 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 09:17:47.721194 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:47.720850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:48843)
0811 09:17:47.720996 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:47.721065 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 09:17:47.721400 10328 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:17:48.044333 10332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:48.044595 10332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:48.044696 10332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:48.051653 10332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:48.051745 10332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:48.051860 10332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:17:48.061235 10332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:35133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=46291
--tserver_master_addrs=127.17.16.126:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:48.062434 10332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:48.063697 10332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:48.075944 10342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:48.076004 10339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:48.076027 10338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:48.077023 10332 server_base.cc:1034] running on GCE node
I20260811 09:17:48.077538 10332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:48.078336 10332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:48.079522 10332 hybrid_clock.cc:648] HybridClock initialized: now 1786439868079473 us; error 42 us; skew 500 ppm
I20260811 09:17:48.079775 10332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:48 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439868, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:48.082764 10332 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:17:48.084117 10332 webserver.cc:460] Webserver started at http://127.17.16.66:46291/ using document root <none> and password file <none>
I20260811 09:17:48.084740 10332 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:48.084864 10332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:48.089871 10332 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:17:48.092346 10357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:48.093583 10332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 09:17:48.093727 10332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "be1b77a6ada0496f96ea1cbe1ef17f42"
format_stamp: "Formatted at 2026-08-11 09:17:42 on dist-test-slave-ncp9"
server_key: "2356194807b09d431df3655bf32de647"
server_key_iv: "d07fda8a4c287fa5f360bd62bbc1f0d9"
server_key_version: "encryptionkey@0"
I20260811 09:17:48.094655 10332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:48.108613 10332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:48.112432 10332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:48.112681 10332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:48.113619 10332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:48.114842 10332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:48.114938 10332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:48.115044 10332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:48.115118 10332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:48.144661 10332 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:35133
I20260811 09:17:48.144714 10529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:35133 every 8 connection(s)
I20260811 09:17:48.146090 10332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:17:48.150564 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:48.148818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:56823)
0811 09:17:48.149552 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:48.149758 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":277,"threads_started":1}
W20260811 09:17:48.151144 10532 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:37471 (0 consecutive failures): Network error: Failed to ping master at 127.17.16.126:37471: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
I20260811 09:17:48.151940 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 10332
I20260811 09:17:48.151981 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:48.151614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:53547)
0811 09:17:48.151770 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:48.151853 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 09:17:48.152736 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:42909
--local_ip_for_outbound_sockets=127.17.16.67
--tserver_master_addrs=127.17.16.126:37471
--webserver_port=40235
--webserver_interface=127.17.16.67
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:17:48.152871 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:48.152514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:44645)
0811 09:17:48.152647 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:48.152737 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 09:17:48.153127 10532 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:17:48.473635 10538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:17:48.473875 10538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:17:48.474005 10538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:17:48.480885 10538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:17:48.481001 10538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:17:48.481156 10538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:17:48.490353 10538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:42909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=40235
--tserver_master_addrs=127.17.16.126:37471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:17:48.491520 10538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:17:48.492762 10538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:17:48.505386 10548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:48.505352 10549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:17:48.505300 10552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:17:48.506196 10538 server_base.cc:1034] running on GCE node
I20260811 09:17:48.506788 10538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:17:48.507627 10538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:17:48.508818 10538 hybrid_clock.cc:648] HybridClock initialized: now 1786439868508781 us; error 30 us; skew 500 ppm
I20260811 09:17:48.509115 10538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:17:48 dist-test-slave-ncp9 krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439868, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:17:48.511994 10538 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:17:48.513321 10538 webserver.cc:460] Webserver started at http://127.17.16.67:40235/ using document root <none> and password file <none>
I20260811 09:17:48.513927 10538 fs_manager.cc:359] Metadata directory not provided
I20260811 09:17:48.514027 10538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:17:48.518998 10538 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 09:17:48.521535 10560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:48.522769 10538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:17:48.522902 10538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "bf2c040634154395bc16c3879da07551"
format_stamp: "Formatted at 2026-08-11 09:17:42 on dist-test-slave-ncp9"
server_key: "cf2a47a5d6657ae872ae22931f59c31e"
server_key_iv: "a939ba571e330ced749dd005a0fc3a22"
server_key_version: "encryptionkey@0"
I20260811 09:17:48.523845 10538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:17:48.540438 10538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:17:48.544094 10538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:17:48.544345 10538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:17:48.545282 10538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:17:48.546525 10538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:17:48.546618 10538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:48.546725 10538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:17:48.546795 10538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:17:48.576346 10538 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:42909
I20260811 09:17:48.576406 10739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:42909 every 8 connection(s)
I20260811 09:17:48.577721 10538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:17:48.578006 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 10538
I20260811 09:17:48.582281 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:48.580590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:34757)
0811 09:17:48.581283 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:48.581481 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":288,"threads_started":1}
W20260811 09:17:48.582952 10741 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:37471 (0 consecutive failures): Network error: Failed to ping master at 127.17.16.126:37471: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
I20260811 09:17:48.583686 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:48.583369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:43373)
0811 09:17:48.583510 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:48.583570 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 09:17:48.584417 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:48.584113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:35419)
0811 09:17:48.584246 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:48.584317 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 09:17:48.584645 10741 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:17:48.722656 10746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:48.721884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:60609)
0811 09:17:48.722328 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:48.722444 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":205,"threads_started":1}
I20260811 09:17:49.154435 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:49.153568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:48789)
0811 09:17:49.154066 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:49.154191 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":242,"threads_started":1}
I20260811 09:17:49.249372 10758 raft_consensus.cc:482] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:17:49.249691 10758 raft_consensus.cc:504] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:49.251284 10758 leader_election.cc:288] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf2c040634154395bc16c3879da07551 (127.17.16.67:42909), be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133)
Aug 11 09:17:49 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439867, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
Aug 11 09:17:49 dist-test-slave-ncp9 krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439867, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:17:49.275652 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:49.251747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35133 (local address 127.17.16.65:49111)
0811 09:17:49.253687 (+ 1940us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:49.253707 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:49.253825 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:49.258163 (+ 4338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:49.258175 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:49.259108 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:49.259984 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:49.260014 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:49.261914 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:49.261920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:49.263627 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:49.263642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:49.264737 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:49.264745 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:49.265072 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:49.266363 (+ 1291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:49.266382 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:49.268495 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:49.271739 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:49.271748 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:49.271761 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:49.272341 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:49.272853 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:49.272858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:49.272862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:49.273177 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:49.274373 (+ 1196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:49.274377 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:49.274529 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 09:17:49.274704 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 09:17:49.275117 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1794,"mutex_wait_us":69,"spinlock_wait_cycles":28288,"thread_start_us":1804,"threads_started":1}
I20260811 09:17:49.275952 10761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:49.251835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:39737 (local address 127.17.16.67:42909)
0811 09:17:49.257272 (+ 5437us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:49.257282 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:49.257313 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:17:49.259313 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:49.259324 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:49.259761 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:49.260136 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:49.261172 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:49.262744 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:49.264105 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:49.264866 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:49.265073 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:49.268578 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:49.268587 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:49.268599 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:49.268643 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:49.271600 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:49.272468 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:49.272477 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:49.272483 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:49.272692 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:49.273355 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:49.273360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:49.273364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:49.274078 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:49.274271 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:49.275160 (+ 889us) server_negotiation.cc:299] Negotiation successful
0811 09:17:49.275505 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5289,"thread_start_us":5163,"threads_started":1}
I20260811 09:17:49.276171 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:49.252245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:49111 (local address 127.17.16.66:35133)
0811 09:17:49.254896 (+ 2651us) server_negotiation.cc:206] Beginning negotiation
0811 09:17:49.254912 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:17:49.254970 (+ 58us) server_negotiation.cc:407] Connection header received
0811 09:17:49.257325 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:17:49.257334 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:17:49.257685 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:17:49.257939 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:17:49.260210 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:49.261726 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:49.263772 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:17:49.264513 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:17:49.264637 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:49.268745 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:17:49.268751 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:17:49.268763 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:17:49.268797 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:17:49.271598 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:49.272497 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:49.272505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:49.272512 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:49.272713 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:17:49.273322 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:17:49.273328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:17:49.273332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:17:49.274078 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:17:49.274271 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 09:17:49.275017 (+ 746us) server_negotiation.cc:299] Negotiation successful
0811 09:17:49.275431 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2439,"thread_start_us":2159,"threads_started":1}
I20260811 09:17:49.275660 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:49.251688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:42909 (local address 127.17.16.65:39737)
0811 09:17:49.252242 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:49.252284 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:17:49.252935 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:17:49.260091 (+ 7156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:17:49.260096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:17:49.260697 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:17:49.261021 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:49.261035 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:49.262913 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:49.262917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:49.263968 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:17:49.263982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:17:49.265094 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:17:49.265101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:17:49.265264 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:17:49.265868 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:17:49.265895 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:17:49.268281 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:17:49.271748 (+ 3467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:49.271754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:49.271762 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:49.272322 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:49.272847 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:17:49.272854 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:17:49.272858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:17:49.273201 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:17:49.274228 (+ 1027us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:17:49.274234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:17:49.274491 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 09:17:49.274701 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 09:17:49.275251 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":145,"spinlock_wait_cycles":4992,"thread_start_us":797,"threads_started":1}
W20260811 09:17:49.279477 10149 leader_election.cc:341] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf2c040634154395bc16c3879da07551 (127.17.16.67:42909): Not found: Tablet not found: fa1610b79ebe4dd3ac8aac37100dc14d
W20260811 09:17:49.279726 10146 leader_election.cc:341] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133): Not found: Tablet not found: fa1610b79ebe4dd3ac8aac37100dc14d
I20260811 09:17:49.279907 10146 leader_election.cc:302] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c007f901f0684f17a8b47286b2538651; no voters: be1b77a6ada0496f96ea1cbe1ef17f42, bf2c040634154395bc16c3879da07551
I20260811 09:17:49.280376 10758 raft_consensus.cc:2726] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:17:49.586011 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:49.585167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:36515)
0811 09:17:49.585626 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:49.585784 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":219,"threads_started":1}
I20260811 09:17:49.723922 10759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:49.723537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:55793)
0811 09:17:49.723694 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:49.723781 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 09:17:50.155946 10765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:50.155289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:45523)
0811 09:17:50.155708 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:50.155794 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":223,"threads_started":1}
I20260811 09:17:50.587488 10766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:50.586847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:47817)
0811 09:17:50.587260 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:50.587338 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":222,"threads_started":1}
I20260811 09:17:50.725498 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:50.724741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:46365)
0811 09:17:50.725233 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:50.725311 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":241,"threads_started":1}
I20260811 09:17:50.798846 10770 raft_consensus.cc:482] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:17:50.799045 10770 raft_consensus.cc:504] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:50.799782 10770 leader_election.cc:288] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf2c040634154395bc16c3879da07551 (127.17.16.67:42909), be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133)
W20260811 09:17:50.800813 10146 leader_election.cc:341] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133): Not found: Tablet not found: fa1610b79ebe4dd3ac8aac37100dc14d
W20260811 09:17:50.801002 10149 leader_election.cc:341] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf2c040634154395bc16c3879da07551 (127.17.16.67:42909): Not found: Tablet not found: fa1610b79ebe4dd3ac8aac37100dc14d
I20260811 09:17:50.801158 10149 leader_election.cc:302] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c007f901f0684f17a8b47286b2538651; no voters: be1b77a6ada0496f96ea1cbe1ef17f42, bf2c040634154395bc16c3879da07551
I20260811 09:17:50.801402 10770 raft_consensus.cc:2726] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:17:51.157518 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:51.156798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:60317)
0811 09:17:51.157279 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:51.157358 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":266,"threads_started":1}
I20260811 09:17:51.589150 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:51.588300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:51545)
0811 09:17:51.588826 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:51.588974 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":257,"threads_started":1}
I20260811 09:17:51.726989 10773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:51.726282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:44171)
0811 09:17:51.726739 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:51.726828 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":216,"threads_started":1}
I20260811 09:17:52.159080 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:52.158298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:40117)
0811 09:17:52.158788 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:52.158878 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":249,"threads_started":1}
I20260811 09:17:52.590826 10778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:52.590057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:56125)
0811 09:17:52.590538 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:52.590629 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":222,"threads_started":1}
I20260811 09:17:52.728645 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:52.727865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:34131)
0811 09:17:52.728357 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:52.728439 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":228,"threads_started":1}
I20260811 09:17:53.160600 10782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:53.159917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:50289)
0811 09:17:53.160362 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:53.160441 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":258,"threads_started":1}
I20260811 09:17:53.592355 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:53.591699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:53871)
0811 09:17:53.592115 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:53.592192 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":225,"threads_started":1}
I20260811 09:17:53.730193 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:53.729494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:56265)
0811 09:17:53.729932 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:53.730018 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":220,"threads_started":1}
I20260811 09:17:53.747107 10786 raft_consensus.cc:482] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:17:53.747295 10786 raft_consensus.cc:504] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:53.747952 10786 leader_election.cc:288] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf2c040634154395bc16c3879da07551 (127.17.16.67:42909), be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133)
W20260811 09:17:53.748867 10149 leader_election.cc:341] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf2c040634154395bc16c3879da07551 (127.17.16.67:42909): Not found: Tablet not found: fa1610b79ebe4dd3ac8aac37100dc14d
W20260811 09:17:53.749085 10146 leader_election.cc:341] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133): Not found: Tablet not found: fa1610b79ebe4dd3ac8aac37100dc14d
I20260811 09:17:53.749240 10146 leader_election.cc:302] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c007f901f0684f17a8b47286b2538651; no voters: be1b77a6ada0496f96ea1cbe1ef17f42, bf2c040634154395bc16c3879da07551
I20260811 09:17:53.749470 10786 raft_consensus.cc:2726] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:17:54.162106 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:54.161419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:56831)
0811 09:17:54.161832 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:54.161914 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":213,"threads_started":1}
I20260811 09:17:54.593911 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:54.593224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:60239)
0811 09:17:54.593669 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:54.593746 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":238,"threads_started":1}
I20260811 09:17:54.731715 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:54.730966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:57149)
0811 09:17:54.731429 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:54.731519 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":254,"threads_started":1}
I20260811 09:17:55.163745 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:55.162948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:53677)
0811 09:17:55.163473 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:55.163563 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":239,"threads_started":1}
I20260811 09:17:55.595506 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:55.594746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:53377)
0811 09:17:55.595241 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:55.595323 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":235,"threads_started":1}
I20260811 09:17:55.728471 10798 raft_consensus.cc:482] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:17:55.728657 10798 raft_consensus.cc:504] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:17:55.729238 10798 leader_election.cc:288] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf2c040634154395bc16c3879da07551 (127.17.16.67:42909), be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133)
W20260811 09:17:55.730101 10149 leader_election.cc:341] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf2c040634154395bc16c3879da07551 (127.17.16.67:42909): Not found: Tablet not found: fa1610b79ebe4dd3ac8aac37100dc14d
W20260811 09:17:55.730259 10146 leader_election.cc:341] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133): Not found: Tablet not found: fa1610b79ebe4dd3ac8aac37100dc14d
I20260811 09:17:55.730396 10146 leader_election.cc:302] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c007f901f0684f17a8b47286b2538651; no voters: be1b77a6ada0496f96ea1cbe1ef17f42, bf2c040634154395bc16c3879da07551
I20260811 09:17:55.730665 10798 raft_consensus.cc:2726] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:17:55.733196 10799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:55.732472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:46659)
0811 09:17:55.732919 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:55.733034 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":215,"threads_started":1}
I20260811 09:17:56.165305 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:56.164589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:54247)
0811 09:17:56.165036 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:56.165124 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":212,"threads_started":1}
I20260811 09:17:56.597117 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:56.596321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:32805)
0811 09:17:56.596783 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:56.596872 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":234,"threads_started":1}
I20260811 09:17:56.734643 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:56.733919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:36537)
0811 09:17:56.734359 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:56.734438 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":226,"threads_started":1}
I20260811 09:17:57.166822 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:57.166107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:40633)
0811 09:17:57.166581 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:57.166666 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":214,"threads_started":1}
I20260811 09:17:57.598753 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:57.597995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:35801)
0811 09:17:57.598489 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:57.598594 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":219,"threads_started":1}
I20260811 09:17:57.739135 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:57.737242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:35479)
0811 09:17:57.738783 (+ 1541us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:57.738897 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1433,"thread_start_us":1331,"threads_started":1}
I20260811 09:17:58.168340 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:58.167649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:59789)
0811 09:17:58.168088 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:58.168166 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":233,"threads_started":1}
I20260811 09:17:58.600258 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:58.599541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:50029)
0811 09:17:58.599991 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:58.600075 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":229,"threads_started":1}
I20260811 09:17:58.740831 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:58.740109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:35035)
0811 09:17:58.740554 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:58.740642 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":220,"threads_started":1}
I20260811 09:17:59.169883 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:59.169124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:53699)
0811 09:17:59.169635 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:59.169717 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":248,"threads_started":1}
I20260811 09:17:59.601984 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:59.601203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:38899)
0811 09:17:59.601720 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:59.601811 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":248,"threads_started":1}
I20260811 09:17:59.742602 10819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:17:59.741836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:46667)
0811 09:17:59.742341 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 09:17:59.742437 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":255,"threads_started":1}
I20260811 09:18:00.171450 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:00.170772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:33393)
0811 09:18:00.171199 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:00.171281 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":221,"threads_started":1}
I20260811 09:18:00.603508 10821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:00.602806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:38781)
0811 09:18:00.603266 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:00.603344 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":243,"threads_started":1}
I20260811 09:18:00.744241 10826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:00.743512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:47355)
0811 09:18:00.743955 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:00.744067 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":234,"threads_started":1}
I20260811 09:18:01.173110 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:01.172317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:54227)
0811 09:18:01.172801 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:01.172930 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":229,"threads_started":1}
I20260811 09:18:01.605065 10828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:01.604337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:56043)
0811 09:18:01.604776 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:01.604863 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":230,"threads_started":1}
I20260811 09:18:01.741329 10829 raft_consensus.cc:482] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:18:01.741504 10829 raft_consensus.cc:504] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c007f901f0684f17a8b47286b2538651" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 45591 } } peers { permanent_uuid: "bf2c040634154395bc16c3879da07551" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42909 } } peers { permanent_uuid: "be1b77a6ada0496f96ea1cbe1ef17f42" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35133 } }
I20260811 09:18:01.742110 10829 leader_election.cc:288] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf2c040634154395bc16c3879da07551 (127.17.16.67:42909), be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133)
W20260811 09:18:01.743005 10149 leader_election.cc:341] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf2c040634154395bc16c3879da07551 (127.17.16.67:42909): Not found: Tablet not found: fa1610b79ebe4dd3ac8aac37100dc14d
W20260811 09:18:01.743177 10146 leader_election.cc:341] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be1b77a6ada0496f96ea1cbe1ef17f42 (127.17.16.66:35133): Not found: Tablet not found: fa1610b79ebe4dd3ac8aac37100dc14d
I20260811 09:18:01.743330 10146 leader_election.cc:302] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c007f901f0684f17a8b47286b2538651; no voters: be1b77a6ada0496f96ea1cbe1ef17f42, bf2c040634154395bc16c3879da07551
I20260811 09:18:01.743556 10829 raft_consensus.cc:2726] T fa1610b79ebe4dd3ac8aac37100dc14d P c007f901f0684f17a8b47286b2538651 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:18:01.745739 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:01.744978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:39389)
0811 09:18:01.745448 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:01.745539 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":251,"threads_started":1}
I20260811 09:18:02.174688 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:02.173956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:45809)
0811 09:18:02.174397 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:02.174475 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":226,"threads_started":1}
I20260811 09:18:02.606693 10833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:02.605900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:35863)
0811 09:18:02.606374 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:02.606466 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":239,"threads_started":1}
I20260811 09:18:02.747709 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:02.746917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.65:34321)
0811 09:18:02.747432 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:02.747519 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":286,"threads_started":1}
I20260811 09:18:03.176447 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:03.175740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.66:48001)
0811 09:18:03.176163 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:03.176288 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":203,"threads_started":1}
I20260811 09:18:03.586270 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 10055
I20260811 09:18:03.601073 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:03.587375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52172 (local address 127.17.16.65:45591)
0811 09:18:03.587878 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:03.587887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:03.587915 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:18:03.587978 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:03.587986 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:03.588305 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:03.588497 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:03.590496 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:03.591473 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:03.592624 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:03.593249 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:03.593347 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:03.595767 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:03.595779 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:03.595794 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:03.595831 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:03.598555 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:03.599238 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:03.599246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:03.599248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:03.599379 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:03.599663 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:03.599666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:03.599668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:03.600007 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:03.600232 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:03.600499 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 09:18:03.600712 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":668,"threads_started":1}
I20260811 09:18:03.602279 10319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:03.608417 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:03.607692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37471 (local address 127.17.16.67:52231)
0811 09:18:03.608154 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:03.608273 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":228,"threads_started":1}
I20260811 09:18:03.689765 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 10055
I20260811 09:18:03.713100 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 10332
I20260811 09:18:03.726289 10843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:03.713897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43512 (local address 127.17.16.66:35133)
0811 09:18:03.714342 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:03.714346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:03.714364 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:18:03.714422 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:03.714426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:03.714484 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:03.714576 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:03.716103 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:03.716847 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:03.718123 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:03.718557 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:03.718647 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:03.721500 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:03.721505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:03.721509 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:03.721546 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:03.724103 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:03.724693 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:03.724696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:03.724698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:03.724819 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:03.725156 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:03.725159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:03.725161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:03.725424 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:03.725622 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:03.725824 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 09:18:03.725991 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":244,"threads_started":1}
I20260811 09:18:03.727073 10524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:03.809100 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 10332
I20260811 09:18:03.830691 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 10538
I20260811 09:18:03.843596 10845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:03.831474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.17.16.67:42909)
0811 09:18:03.832035 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:03.832040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:03.832058 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:18:03.832116 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:03.832120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:03.832196 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:03.832284 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:03.833681 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:03.834487 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:03.835693 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:03.836128 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:03.836238 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:03.838758 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:03.838764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:03.838767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:03.838801 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:03.841500 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:03.842001 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:03.842004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:03.842007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:03.842162 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:03.842439 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:03.842442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:03.842443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:03.842679 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:03.842874 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:03.843083 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 09:18:03.843238 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":290,"threads_started":1}
I20260811 09:18:03.844269 10730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:03.925132 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 10538
2026-08-11T09:18:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24467 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:18:04 dist-test-slave-ncp9 krb5kdc[10850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:18:04 dist-test-slave-ncp9 krb5kdc[10850](info): set up 2 sockets
Aug 11 09:18:04 dist-test-slave-ncp9 krb5kdc[10850](info): commencing operation
krb5kdc: starting...
W20260811 09:18:05.156291 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:18:05 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:18:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:18:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:18:05.433370 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:35519
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:35519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:18:05.773327 10867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:05.773629 10867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:05.773775 10867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:05.781512 10867 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 09:18:05.781620 10867 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 09:18:05.781687 10867 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 09:18:05.781766 10867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:18:05.781847 10867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:05.781960 10867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:05.782019 10867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:18:05.782104 10867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:18:05.792090 10867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:35519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:35519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:05.793381 10867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:05.794759 10867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:05.806933 10875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:05.806892 10876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:05.806978 10881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:05.808014 10867 server_base.cc:1034] running on GCE node
I20260811 09:18:05.808821 10867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:05.810215 10867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:05.811436 10867 hybrid_clock.cc:648] HybridClock initialized: now 1786439885811367 us; error 43 us; skew 500 ppm
I20260811 09:18:05.811750 10867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:05 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:05.814739 10867 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:18:05.816175 10867 webserver.cc:460] Webserver started at http://127.17.16.126:41993/ using document root <none> and password file <none>
I20260811 09:18:05.816823 10867 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:05.816955 10867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:05.817265 10867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:18:05.820084 10867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6"
format_stamp: "Formatted at 2026-08-11 09:18:05 on dist-test-slave-ncp9"
server_key: "1cff16cfcf17a05d1da46cfcae5921f3"
server_key_iv: "50d31734667d2cc9106fbe449ba29108"
server_key_version: "encryptionkey@0"
I20260811 09:18:05.820837 10867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6"
format_stamp: "Formatted at 2026-08-11 09:18:05 on dist-test-slave-ncp9"
server_key: "1cff16cfcf17a05d1da46cfcae5921f3"
server_key_iv: "50d31734667d2cc9106fbe449ba29108"
server_key_version: "encryptionkey@0"
I20260811 09:18:05.825734 10867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 09:18:05.828943 10890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:05.830209 10867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:18:05.830363 10867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6"
format_stamp: "Formatted at 2026-08-11 09:18:05 on dist-test-slave-ncp9"
server_key: "1cff16cfcf17a05d1da46cfcae5921f3"
server_key_iv: "50d31734667d2cc9106fbe449ba29108"
server_key_version: "encryptionkey@0"
I20260811 09:18:05.830549 10867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:05.856824 10867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:05.860920 10867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:05.861182 10867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:05.879204 10867 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:35519
I20260811 09:18:05.879232 10981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:35519 every 8 connection(s)
I20260811 09:18:05.880511 10867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:18:05.884374 10982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:18:05.886464 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 10867
I20260811 09:18:05.886721 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:18:05.887655 17473 external_mini_cluster.cc:1428] Setting key 36d53ce5e53d8a77378e46d684730bd9
I20260811 09:18:05.897644 10982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Bootstrap starting.
I20260811 09:18:05.901930 10982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Neither blocks nor log segments found. Creating new log.
Aug 11 09:18:05 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:18:05.903129 10982 log.cc:824] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Log is configured to *not* fsync() on all Append() calls
I20260811 09:18:05.906080 10982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: No bootstrap required, opened a new log
I20260811 09:18:05.911315 10987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:05.889902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58122 (local address 127.17.16.126:35519)
0811 09:18:05.890667 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:05.890678 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:05.890721 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:18:05.892352 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:05.892375 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:05.892735 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:05.893073 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:05.894957 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:05.896346 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:05.898345 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:05.898994 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:05.899116 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:05.904326 (+ 5210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:05.904336 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:05.904355 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:05.904401 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:05.907322 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:05.908124 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:05.908130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:05.908143 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:05.908315 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:05.908716 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:05.908719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:05.908721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:05.909290 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:05.909521 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:05.909976 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 09:18:05.910317 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":338,"threads_started":1}
I20260811 09:18:05.914474 10982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:05.914800 10982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:18:05.914898 10982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ffe6e12e0de4c74b3cbb39027e7b5e6, State: Initialized, Role: FOLLOWER
I20260811 09:18:05.915411 10982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:05.915627 10982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:18:05.915752 10982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:18:05.915896 10982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:18:05.918102 10982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:05.918574 10982 leader_election.cc:302] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ffe6e12e0de4c74b3cbb39027e7b5e6; no voters:
I20260811 09:18:05.919090 10982 leader_election.cc:288] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:18:05.919198 10991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:18:05.920252 10991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ffe6e12e0de4c74b3cbb39027e7b5e6, State: Running, Role: LEADER
I20260811 09:18:05.920799 10991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:05.921176 10982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:18:05.923780 10993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } } }
I20260811 09:18:05.923812 10994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ffe6e12e0de4c74b3cbb39027e7b5e6. Latest consensus state: current_term: 1 leader_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } } }
I20260811 09:18:05.924193 10993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: This master's current role is: LEADER
I20260811 09:18:05.924197 10994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: This master's current role is: LEADER
I20260811 09:18:05.929719 11003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:18:05.934700 11003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:18:05.944108 11003 catalog_manager.cc:1347] Generated new cluster ID: 018d493d4e30478d81d4042c57d09195
I20260811 09:18:05.944226 11003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:18:05.964254 11003 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:18:05.965713 11003 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:18:05.977108 11003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Generated new TSK 0
I20260811 09:18:05.978178 11003 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:18:06.050485 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:35519
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:18:06.381315 11023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:06.381592 11023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:06.381736 11023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:06.388645 11023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:06.388758 11023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:06.388931 11023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:18:06.398298 11023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:35519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:06.399513 11023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:06.400772 11023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:06.413697 11031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:06.413666 11032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:06.413630 11036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:06.414326 11023 server_base.cc:1034] running on GCE node
I20260811 09:18:06.414930 11023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:06.415782 11023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:06.416994 11023 hybrid_clock.cc:648] HybridClock initialized: now 1786439886416950 us; error 29 us; skew 500 ppm
I20260811 09:18:06.417302 11023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:06 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:06.420295 11023 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:18:06.421691 11023 webserver.cc:460] Webserver started at http://127.17.16.65:35169/ using document root <none> and password file <none>
I20260811 09:18:06.422377 11023 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:06.422513 11023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:06.422818 11023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:18:06.425833 11023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "2dec3e1860984249950375bf6012fbdd"
format_stamp: "Formatted at 2026-08-11 09:18:06 on dist-test-slave-ncp9"
server_key: "b1419bda2b7fd87b54f9526a81d7afdd"
server_key_iv: "673877bed89e8caaff90bcf8f107a21d"
server_key_version: "encryptionkey@0"
I20260811 09:18:06.426554 11023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "2dec3e1860984249950375bf6012fbdd"
format_stamp: "Formatted at 2026-08-11 09:18:06 on dist-test-slave-ncp9"
server_key: "b1419bda2b7fd87b54f9526a81d7afdd"
server_key_iv: "673877bed89e8caaff90bcf8f107a21d"
server_key_version: "encryptionkey@0"
I20260811 09:18:06.431546 11023 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:18:06.434770 11050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:06.435966 11023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:18:06.436126 11023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "2dec3e1860984249950375bf6012fbdd"
format_stamp: "Formatted at 2026-08-11 09:18:06 on dist-test-slave-ncp9"
server_key: "b1419bda2b7fd87b54f9526a81d7afdd"
server_key_iv: "673877bed89e8caaff90bcf8f107a21d"
server_key_version: "encryptionkey@0"
I20260811 09:18:06.436304 11023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:06.455626 11023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:06.459262 11023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:06.459513 11023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:06.460404 11023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:18:06.461666 11023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:18:06.461748 11023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:06.461864 11023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:18:06.461918 11023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:06.491300 11023 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:42121
I20260811 09:18:06.491346 11236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:42121 every 8 connection(s)
I20260811 09:18:06.492497 11023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:18:06.498593 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 11023
I20260811 09:18:06.498785 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:18:06.499414 17473 external_mini_cluster.cc:1428] Setting key 9b6bb1f00155f2517ed37840abfd85f7
Aug 11 09:18:06 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:18:06.511332 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:06.494933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:39907 (local address 127.17.16.126:35519)
0811 09:18:06.495348 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:06.495353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:06.496151 (+ 798us) server_negotiation.cc:407] Connection header received
0811 09:18:06.498051 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:06.498056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:06.498129 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:06.498225 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:06.499755 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:06.500483 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:06.502036 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:06.502450 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:06.502583 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:06.506014 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:06.506026 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:06.506030 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:06.506062 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:06.508395 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:06.509114 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:06.509118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:06.509120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:06.509245 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:06.509664 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:06.509668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:06.509670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:06.509929 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:06.510072 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:06.510753 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 09:18:06.510980 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":219,"threads_started":1}
I20260811 09:18:06.512213 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:06.495327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:39907)
0811 09:18:06.496002 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:06.496042 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:06.497765 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:06.498466 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:06.498473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:06.499019 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:18:06.499591 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:06.499612 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:06.500650 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:06.500654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:06.501893 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:06.501905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:06.502685 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:06.502693 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:06.502922 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:06.503579 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:18:06.503606 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:18:06.505815 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:18:06.508530 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:06.508539 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:06.508554 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:06.508999 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:06.509351 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:06.509356 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:06.509361 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:06.509552 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:06.510054 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:18:06.510062 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:18:06.510375 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 09:18:06.510610 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 09:18:06.511122 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":252,"threads_started":1}
I20260811 09:18:06.513724 11237 heartbeater.cc:347] Connected to a master server at 127.17.16.126:35519
I20260811 09:18:06.514003 11237 heartbeater.cc:464] Registering TS with master...
I20260811 09:18:06.514693 11237 heartbeater.cc:511] Master 127.17.16.126:35519 requested a full tablet report, sending...
I20260811 09:18:06.516655 10921 ts_manager.cc:194] Registered new tserver with Master: 2dec3e1860984249950375bf6012fbdd (127.17.16.65:42121)
I20260811 09:18:06.518105 10921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:39907
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:18:06.558270 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:35519
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:18:06.882380 11246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:06.882655 11246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:06.882797 11246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:06.890093 11246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:06.890219 11246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:06.890374 11246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:18:06.899863 11246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:35519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:06.901125 11246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:06.902401 11246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:06.914955 11257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:06.915097 11260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:06.914937 11255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:06.915661 11246 server_base.cc:1034] running on GCE node
I20260811 09:18:06.916178 11246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:06.917022 11246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:06.918216 11246 hybrid_clock.cc:648] HybridClock initialized: now 1786439886918170 us; error 39 us; skew 500 ppm
I20260811 09:18:06.918511 11246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:06 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:06.921473 11246 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:18:06.922829 11246 webserver.cc:460] Webserver started at http://127.17.16.66:44375/ using document root <none> and password file <none>
I20260811 09:18:06.923439 11246 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:06.923537 11246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:06.923791 11246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:18:06.926548 11246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "b92630152cec469ebc7118282dfbeee2"
format_stamp: "Formatted at 2026-08-11 09:18:06 on dist-test-slave-ncp9"
server_key: "50eb4e6a11ce4f8715677484065d1640"
server_key_iv: "a2ec63345a23b0bfe81a2337a36c8e64"
server_key_version: "encryptionkey@0"
I20260811 09:18:06.927239 11246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "b92630152cec469ebc7118282dfbeee2"
format_stamp: "Formatted at 2026-08-11 09:18:06 on dist-test-slave-ncp9"
server_key: "50eb4e6a11ce4f8715677484065d1640"
server_key_iv: "a2ec63345a23b0bfe81a2337a36c8e64"
server_key_version: "encryptionkey@0"
I20260811 09:18:06.932138 11246 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:18:06.935612 11271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:06.936828 11246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:18:06.936986 11246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "b92630152cec469ebc7118282dfbeee2"
format_stamp: "Formatted at 2026-08-11 09:18:06 on dist-test-slave-ncp9"
server_key: "50eb4e6a11ce4f8715677484065d1640"
server_key_iv: "a2ec63345a23b0bfe81a2337a36c8e64"
server_key_version: "encryptionkey@0"
I20260811 09:18:06.937155 11246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:07.001529 11246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:07.005846 11246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:07.006101 11246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:07.007040 11246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:18:07.008446 11246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:18:07.008525 11246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:07.008615 11246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:18:07.008666 11246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:07.038594 11246 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:34405
I20260811 09:18:07.038658 11455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:34405 every 8 connection(s)
I20260811 09:18:07.039893 11246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:18:07.043190 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 11246
I20260811 09:18:07.043375 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:18:07.043838 17473 external_mini_cluster.cc:1428] Setting key 7ac164403be465ad3f4d5eae2c773c6a
Aug 11 09:18:07 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:18:07.060217 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.042410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:58593 (local address 127.17.16.126:35519)
0811 09:18:07.042892 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:07.042899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:07.043867 (+ 968us) server_negotiation.cc:407] Connection header received
0811 09:18:07.045766 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:07.045772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:07.045841 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:07.045929 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:07.047745 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.048426 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.050417 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.050817 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.050908 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.054720 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:07.054725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:07.054729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:07.054760 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:07.057280 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:07.058027 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:07.058031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:07.058033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:07.058157 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:07.058558 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:07.058561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:07.058563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:07.058883 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:07.059023 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:07.059633 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 09:18:07.059869 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":200,"threads_started":1}
I20260811 09:18:07.060909 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.042939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:58593)
0811 09:18:07.043696 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:07.043742 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:07.045520 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:07.046207 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:07.046219 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:07.046870 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:18:07.047599 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.047623 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.048573 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.048578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.050302 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.050316 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.051031 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.051037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.051310 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.051959 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:18:07.051994 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:18:07.054496 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:18:07.057403 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:07.057410 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:07.057424 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:07.057917 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:07.058253 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:07.058258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:07.058261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:07.058470 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:07.058995 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:18:07.059000 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:18:07.059291 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 09:18:07.059507 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 09:18:07.059869 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":293,"threads_started":1}
I20260811 09:18:07.062227 11456 heartbeater.cc:347] Connected to a master server at 127.17.16.126:35519
I20260811 09:18:07.062526 11456 heartbeater.cc:464] Registering TS with master...
I20260811 09:18:07.063357 11456 heartbeater.cc:511] Master 127.17.16.126:35519 requested a full tablet report, sending...
I20260811 09:18:07.064874 10921 ts_manager.cc:194] Registered new tserver with Master: b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
I20260811 09:18:07.065744 10921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:58593
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:18:07.108392 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:35519
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:18:07.436048 11464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:07.436306 11464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:07.436436 11464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:07.444106 11464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:07.444239 11464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:07.444423 11464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:18:07.455214 11464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:35519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:07.456451 11464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:07.457736 11464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:07.469947 11480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:07.470001 11474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:07.470037 11475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:07.470810 11464 server_base.cc:1034] running on GCE node
I20260811 09:18:07.471375 11464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:07.472220 11464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:07.473423 11464 hybrid_clock.cc:648] HybridClock initialized: now 1786439887473374 us; error 44 us; skew 500 ppm
I20260811 09:18:07.473722 11464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:07 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439887, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:07.476632 11464 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:18:07.477996 11464 webserver.cc:460] Webserver started at http://127.17.16.67:32939/ using document root <none> and password file <none>
I20260811 09:18:07.478627 11464 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:07.478747 11464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:07.479032 11464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:18:07.481889 11464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "b358b25eee3541609d8800dd14f09e48"
format_stamp: "Formatted at 2026-08-11 09:18:07 on dist-test-slave-ncp9"
server_key: "e94e72162ce38f02892dda0dab75b885"
server_key_iv: "7809d691fa7a183ccd680e7112a02e26"
server_key_version: "encryptionkey@0"
I20260811 09:18:07.482611 11464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "b358b25eee3541609d8800dd14f09e48"
format_stamp: "Formatted at 2026-08-11 09:18:07 on dist-test-slave-ncp9"
server_key: "e94e72162ce38f02892dda0dab75b885"
server_key_iv: "7809d691fa7a183ccd680e7112a02e26"
server_key_version: "encryptionkey@0"
I20260811 09:18:07.487459 11464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:18:07.490751 11490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:07.492015 11464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:18:07.492169 11464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "b358b25eee3541609d8800dd14f09e48"
format_stamp: "Formatted at 2026-08-11 09:18:07 on dist-test-slave-ncp9"
server_key: "e94e72162ce38f02892dda0dab75b885"
server_key_iv: "7809d691fa7a183ccd680e7112a02e26"
server_key_version: "encryptionkey@0"
I20260811 09:18:07.492339 11464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:07.507367 11464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:07.510851 11464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:07.511113 11464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:07.511976 11464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:18:07.513254 11464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:18:07.513325 11464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:07.513434 11464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:18:07.513484 11464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:07.520871 11237 heartbeater.cc:503] Master 127.17.16.126:35519 was elected leader, sending a full tablet report...
I20260811 09:18:07.542507 11464 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:34061
I20260811 09:18:07.542547 11672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:34061 every 8 connection(s)
I20260811 09:18:07.543686 11464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:18:07.545154 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 11464
I20260811 09:18:07.545322 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:18:07.545758 17473 external_mini_cluster.cc:1428] Setting key c364583c06c9a528a307f027815f92af
Aug 11 09:18:07 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439887, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:18:07.562124 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.546416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:32901 (local address 127.17.16.126:35519)
0811 09:18:07.546605 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:07.546609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:07.547512 (+ 903us) server_negotiation.cc:407] Connection header received
0811 09:18:07.549364 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:07.549368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:07.549447 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:07.549522 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:07.550846 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.551578 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.553028 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.553416 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.553501 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.556718 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:07.556723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:07.556726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:07.556760 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:07.559276 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:07.559936 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:07.559940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:07.559943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:07.560071 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:07.560492 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:07.560496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:07.560498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:07.560767 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:07.560943 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:07.561567 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 09:18:07.561804 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:18:07.563012 11677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.546530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:32901)
0811 09:18:07.547360 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:07.547406 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:07.549117 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:07.549711 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:07.549719 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:07.550191 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:18:07.550722 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.550739 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.551698 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.551701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.552883 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.552924 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.553582 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.553586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.553798 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.554390 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:18:07.554411 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:18:07.556517 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:18:07.559408 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:07.559417 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:07.559430 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:07.559823 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:07.560181 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:07.560187 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:07.560191 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:07.560383 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:07.560907 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:18:07.560915 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:18:07.561225 (+ 310us) client_negotiation.cc:769] Sending connection context
0811 09:18:07.561448 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 09:18:07.561923 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":327,"threads_started":1}
I20260811 09:18:07.564266 11674 heartbeater.cc:347] Connected to a master server at 127.17.16.126:35519
I20260811 09:18:07.564550 11674 heartbeater.cc:464] Registering TS with master...
I20260811 09:18:07.565253 11674 heartbeater.cc:511] Master 127.17.16.126:35519 requested a full tablet report, sending...
I20260811 09:18:07.566540 10921 ts_manager.cc:194] Registered new tserver with Master: b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061)
I20260811 09:18:07.567297 10921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:32901
I20260811 09:18:07.572499 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:18:07.588300 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.576824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.17.16.126:35519)
0811 09:18:07.577012 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:07.577016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:07.577450 (+ 434us) server_negotiation.cc:407] Connection header received
0811 09:18:07.577587 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:07.577590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:07.577652 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:07.577726 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:07.579114 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.579819 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.581162 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.581559 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.581653 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.583873 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:07.583878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:07.583881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:07.583907 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:07.586220 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:07.586796 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:07.586800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:07.586802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:07.586921 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:07.587206 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:07.587210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:07.587212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:07.587503 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:07.587701 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:07.587916 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 09:18:07.588073 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:18:07.592010 10921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:18:07.598678 10921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:18:07.621024 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.610179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33508 (local address 127.17.16.67:34061)
0811 09:18:07.611225 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:07.611234 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:07.611675 (+ 441us) server_negotiation.cc:407] Connection header received
0811 09:18:07.612060 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:07.612068 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:07.612272 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:07.612394 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:18:07.613204 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.614902 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.617872 (+ 2970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.619387 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.619491 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.619724 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:07.620449 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 09:18:07.620629 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":670,"threads_started":1}
I20260811 09:18:07.622119 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.610023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:34061 (local address 127.0.0.1:33508)
0811 09:18:07.611513 (+ 1490us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:07.611572 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:07.611814 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:07.612665 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:07.612670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:07.612693 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:18:07.613011 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.613022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.615090 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.615096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.617674 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.617691 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.619941 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.619947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.620177 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.620227 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 09:18:07.621538 (+ 1311us) client_negotiation.cc:240] Negotiation successful
0811 09:18:07.621754 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1264,"thread_start_us":3196,"threads_started":1}
I20260811 09:18:07.625152 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.610648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.0.0.1:37918)
0811 09:18:07.613420 (+ 2772us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:07.613437 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:07.613546 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:07.615205 (+ 1659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:07.615210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:07.615241 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:18:07.615756 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.615769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.619032 (+ 3263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.619038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.621282 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.621298 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.624249 (+ 2951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.624256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.624482 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.624525 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 09:18:07.624617 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 09:18:07.624751 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2676,"thread_start_us":648,"threads_started":1}
I20260811 09:18:07.625684 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.611150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.17.16.65:42121)
0811 09:18:07.611733 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:07.611741 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:07.613529 (+ 1788us) server_negotiation.cc:407] Connection header received
0811 09:18:07.613696 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:07.613703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:07.613906 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:07.614102 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:18:07.616145 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.618854 (+ 2709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.621467 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.623976 (+ 2509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.624157 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.624417 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:07.625200 (+ 783us) server_negotiation.cc:299] Negotiation successful
0811 09:18:07.625362 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":755,"threads_started":1}
I20260811 09:18:07.628883 11579 tablet_service.cc:1467] Processing CreateTablet for tablet 50a9790621df4e42b7adfb22c251815c (DEFAULT_TABLE table=test-table [id=2e090742a5ba4375a6b3b1bfaa2f766d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:18:07.628804 11140 tablet_service.cc:1467] Processing CreateTablet for tablet 50a9790621df4e42b7adfb22c251815c (DEFAULT_TABLE table=test-table [id=2e090742a5ba4375a6b3b1bfaa2f766d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:18:07.630203 11579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a9790621df4e42b7adfb22c251815c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:18:07.630264 11140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a9790621df4e42b7adfb22c251815c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:18:07.633424 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.613760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:34405 (local address 127.0.0.1:36216)
0811 09:18:07.622650 (+ 8890us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:07.622668 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:07.622769 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:07.625527 (+ 2758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:07.625531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:07.625555 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:18:07.625693 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.625702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.628819 (+ 3117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.628824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.630580 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.630596 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.632389 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.632395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.632806 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.632841 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:18:07.633004 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 09:18:07.633164 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8795,"thread_start_us":11179,"threads_started":1}
I20260811 09:18:07.634292 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.613842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.17.16.66:34405)
0811 09:18:07.624862 (+ 11020us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:07.624870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:07.624917 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:18:07.625033 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:07.625041 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:07.625257 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:07.625650 (+ 393us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:18:07.626659 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.628658 (+ 1999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.630739 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.632203 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.632344 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.632521 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:07.633764 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0811 09:18:07.633931 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10878,"thread_start_us":10741,"threads_started":1}
I20260811 09:18:07.637755 11349 tablet_service.cc:1467] Processing CreateTablet for tablet 50a9790621df4e42b7adfb22c251815c (DEFAULT_TABLE table=test-table [id=2e090742a5ba4375a6b3b1bfaa2f766d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:18:07.639248 11349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a9790621df4e42b7adfb22c251815c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:18:07.641662 11703 tablet_bootstrap.cc:492] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: Bootstrap starting.
I20260811 09:18:07.644850 11704 tablet_bootstrap.cc:492] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: Bootstrap starting.
I20260811 09:18:07.645399 11703 tablet_bootstrap.cc:654] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: Neither blocks nor log segments found. Creating new log.
I20260811 09:18:07.646435 11703 log.cc:824] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: Log is configured to *not* fsync() on all Append() calls
I20260811 09:18:07.648173 11704 tablet_bootstrap.cc:654] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: Neither blocks nor log segments found. Creating new log.
I20260811 09:18:07.649363 11704 log.cc:824] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: Log is configured to *not* fsync() on all Append() calls
I20260811 09:18:07.649439 11703 tablet_bootstrap.cc:492] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: No bootstrap required, opened a new log
I20260811 09:18:07.649664 11703 ts_tablet_manager.cc:1397] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260811 09:18:07.652299 11704 tablet_bootstrap.cc:492] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: No bootstrap required, opened a new log
I20260811 09:18:07.652568 11704 ts_tablet_manager.cc:1397] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:18:07.653179 11707 tablet_bootstrap.cc:492] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: Bootstrap starting.
I20260811 09:18:07.656589 11707 tablet_bootstrap.cc:654] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: Neither blocks nor log segments found. Creating new log.
I20260811 09:18:07.657756 11707 log.cc:824] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:18:07.658593 11703 raft_consensus.cc:348] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } } peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } }
I20260811 09:18:07.659029 11703 raft_consensus.cc:374] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:18:07.659410 11703 raft_consensus.cc:729] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dec3e1860984249950375bf6012fbdd, State: Initialized, Role: FOLLOWER
I20260811 09:18:07.659996 11703 consensus_queue.cc:260] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } } peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } }
I20260811 09:18:07.660564 11704 raft_consensus.cc:348] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } } peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } }
I20260811 09:18:07.661027 11704 raft_consensus.cc:374] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:18:07.661077 11707 tablet_bootstrap.cc:492] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: No bootstrap required, opened a new log
I20260811 09:18:07.661141 11704 raft_consensus.cc:729] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b358b25eee3541609d8800dd14f09e48, State: Initialized, Role: FOLLOWER
I20260811 09:18:07.661381 11707 ts_tablet_manager.cc:1397] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.005s
I20260811 09:18:07.661609 11703 ts_tablet_manager.cc:1428] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: Time spent starting tablet: real 0.012s user 0.008s sys 0.003s
I20260811 09:18:07.661823 11704 consensus_queue.cc:260] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } } peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } }
I20260811 09:18:07.662904 11674 heartbeater.cc:503] Master 127.17.16.126:35519 was elected leader, sending a full tablet report...
I20260811 09:18:07.663472 11704 ts_tablet_manager.cc:1428] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260811 09:18:07.669893 11707 raft_consensus.cc:348] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } } peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } }
I20260811 09:18:07.670203 11707 raft_consensus.cc:374] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:18:07.670302 11707 raft_consensus.cc:729] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92630152cec469ebc7118282dfbeee2, State: Initialized, Role: FOLLOWER
I20260811 09:18:07.670758 11707 consensus_queue.cc:260] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } } peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } }
I20260811 09:18:07.671777 11456 heartbeater.cc:503] Master 127.17.16.126:35519 was elected leader, sending a full tablet report...
I20260811 09:18:07.672247 11707 ts_tablet_manager.cc:1428] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: Time spent starting tablet: real 0.011s user 0.004s sys 0.006s
W20260811 09:18:07.744779 11240 tablet.cc:2369] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:18:07.792026 11457 tablet.cc:2369] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:18:07.795207 11676 tablet.cc:2369] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:18:07.928817 11709 raft_consensus.cc:482] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:18:07.929190 11709 raft_consensus.cc:504] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } } peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } }
I20260811 09:18:07.930918 11709 leader_election.cc:288] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061), b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
I20260811 09:18:07.937304 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.931413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:49599 (local address 127.17.16.67:34061)
0811 09:18:07.931655 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:07.931662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:07.931887 (+ 225us) server_negotiation.cc:407] Connection header received
0811 09:18:07.932129 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:07.932145 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:07.932231 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:07.932326 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:18:07.932960 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.933758 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.935621 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.936375 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.936528 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.936606 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:07.936883 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 09:18:07.937035 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:18:07.937466 11712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.931155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:34061 (local address 127.17.16.65:49599)
0811 09:18:07.931764 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:07.931781 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:07.931939 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:07.932438 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:07.932443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:07.932471 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:18:07.932795 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.932804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.933888 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.933895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.935475 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.935488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.936573 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.936580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.936738 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.936763 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:18:07.936847 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 09:18:07.937119 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":322,"threads_started":1}
I20260811 09:18:07.938022 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.932196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:45807 (local address 127.17.16.66:34405)
0811 09:18:07.932402 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:07.932408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:07.932620 (+ 212us) server_negotiation.cc:407] Connection header received
0811 09:18:07.932797 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:07.932801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:07.932862 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:07.932975 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:18:07.933528 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.934362 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.936085 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.936850 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.937039 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.937108 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:07.937668 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 09:18:07.937793 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:18:07.938668 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.931890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:34405 (local address 127.17.16.65:45807)
0811 09:18:07.932524 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:07.932540 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:07.932665 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:07.932985 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:07.932990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:07.933014 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:18:07.933369 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.933379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.934512 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.934516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.935928 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:07.935941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.937300 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.937307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:07.937479 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.937510 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:18:07.938212 (+ 702us) client_negotiation.cc:240] Negotiation successful
0811 09:18:07.938338 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":299,"threads_started":1}
I20260811 09:18:07.938547 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a9790621df4e42b7adfb22c251815c" candidate_uuid: "2dec3e1860984249950375bf6012fbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b358b25eee3541609d8800dd14f09e48" is_pre_election: true
I20260811 09:18:07.939164 11605 raft_consensus.cc:2454] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dec3e1860984249950375bf6012fbdd in term 0.
I20260811 09:18:07.939641 11386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a9790621df4e42b7adfb22c251815c" candidate_uuid: "2dec3e1860984249950375bf6012fbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92630152cec469ebc7118282dfbeee2" is_pre_election: true
I20260811 09:18:07.939883 11055 leader_election.cc:302] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dec3e1860984249950375bf6012fbdd, b358b25eee3541609d8800dd14f09e48; no voters:
I20260811 09:18:07.940153 11386 raft_consensus.cc:2454] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dec3e1860984249950375bf6012fbdd in term 0.
I20260811 09:18:07.940311 11709 raft_consensus.cc:2781] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:18:07.940457 11709 raft_consensus.cc:482] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:18:07.940593 11709 raft_consensus.cc:3037] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:18:07.942975 11709 raft_consensus.cc:504] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } } peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } }
I20260811 09:18:07.943567 11709 leader_election.cc:288] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 1 election: Requested vote from peers b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061), b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
I20260811 09:18:07.943979 11605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a9790621df4e42b7adfb22c251815c" candidate_uuid: "2dec3e1860984249950375bf6012fbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b358b25eee3541609d8800dd14f09e48"
I20260811 09:18:07.944036 11386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a9790621df4e42b7adfb22c251815c" candidate_uuid: "2dec3e1860984249950375bf6012fbdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92630152cec469ebc7118282dfbeee2"
I20260811 09:18:07.944196 11605 raft_consensus.cc:3037] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:18:07.944211 11386 raft_consensus.cc:3037] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:18:07.947284 11605 raft_consensus.cc:2454] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dec3e1860984249950375bf6012fbdd in term 1.
I20260811 09:18:07.947284 11386 raft_consensus.cc:2454] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dec3e1860984249950375bf6012fbdd in term 1.
I20260811 09:18:07.947651 11055 leader_election.cc:302] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dec3e1860984249950375bf6012fbdd, b92630152cec469ebc7118282dfbeee2; no voters:
I20260811 09:18:07.947934 11709 raft_consensus.cc:2781] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:18:07.948344 11709 raft_consensus.cc:686] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 1 LEADER]: Becoming Leader. State: Replica: 2dec3e1860984249950375bf6012fbdd, State: Running, Role: LEADER
I20260811 09:18:07.948841 11709 consensus_queue.cc:237] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } } peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } }
I20260811 09:18:07.953301 10920 catalog_manager.cc:5495] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd reported cstate change: term changed from 0 to 1, leader changed from <none> to 2dec3e1860984249950375bf6012fbdd (127.17.16.65). New cstate: current_term: 1 leader_uuid: "2dec3e1860984249950375bf6012fbdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } health_report { overall_health: HEALTHY } } }
I20260811 09:18:07.983912 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:07.979105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37926 (local address 127.17.16.65:42121)
0811 09:18:07.979269 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:07.979274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:07.979291 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:18:07.979422 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:07.979426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:07.979517 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:07.979627 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:18:07.979956 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.980706 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.982114 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:07.982525 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:07.982617 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:07.983000 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:18:07.983501 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:18:07.983606 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:07.983645 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 09:18:07.983707 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 09:18:07.991791 11386 raft_consensus.cc:1261] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2 [term 1 FOLLOWER]: Refusing update from remote peer 2dec3e1860984249950375bf6012fbdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:18:07.991791 11605 raft_consensus.cc:1261] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48 [term 1 FOLLOWER]: Refusing update from remote peer 2dec3e1860984249950375bf6012fbdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:18:07.992653 11709 consensus_queue.cc:1035] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:18:07.992964 11714 consensus_queue.cc:1035] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:18:08.005964 11722 mvcc.cc:205] Tried to move back new op lower bound from 7317257781204725760 to 7317257781042450432. Current Snapshot: MvccSnapshot[applied={T|T < 7317257781204725760}]
I20260811 09:18:08.016922 10920 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:18:08.017235 10920 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:18:08.020912 10920 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Sending DeleteTablet for 3 replicas of tablet 50a9790621df4e42b7adfb22c251815c
I20260811 09:18:08.022123 11140 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a9790621df4e42b7adfb22c251815c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:18:08 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:37918
I20260811 09:18:08.022325 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 11023
I20260811 09:18:08.022401 11349 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a9790621df4e42b7adfb22c251815c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:18:08 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:36216
I20260811 09:18:08.022848 11579 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a9790621df4e42b7adfb22c251815c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:18:08 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:33508
I20260811 09:18:08.033900 11737 tablet_replica.cc:330] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: stopping tablet replica
I20260811 09:18:08.034351 11737 raft_consensus.cc:2229] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:18:08.035718 11737 raft_consensus.cc:2258] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:18:08.037282 11735 tablet_replica.cc:330] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: stopping tablet replica
I20260811 09:18:08.037710 11735 raft_consensus.cc:2229] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:18:08.038417 11735 raft_consensus.cc:2258] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:18:08.040443 11737 ts_tablet_manager.cc:1905] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:18:08 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:18:08.045316 11735 ts_tablet_manager.cc:1905] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:18:08.048511 11737 ts_tablet_manager.cc:1918] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:18:08.048866 11736 tablet_replica.cc:330] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: stopping tablet replica
I20260811 09:18:08.049346 11736 raft_consensus.cc:2229] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:18:08.049944 11736 raft_consensus.cc:2258] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:18:08.050365 11737 log.cc:1196] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/50a9790621df4e42b7adfb22c251815c
I20260811 09:18:08.050946 11737 ts_tablet_manager.cc:1939] T 50a9790621df4e42b7adfb22c251815c P b358b25eee3541609d8800dd14f09e48: Deleting consensus metadata
I20260811 09:18:08.052114 11736 ts_tablet_manager.cc:1905] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:18:08.052973 10891 catalog_manager.cc:4841] TS b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061): tablet 50a9790621df4e42b7adfb22c251815c (table test-table [id=2e090742a5ba4375a6b3b1bfaa2f766d]) successfully deleted
I20260811 09:18:08.053469 11735 ts_tablet_manager.cc:1918] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:18:08.053644 11735 log.cc:1196] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/50a9790621df4e42b7adfb22c251815c
I20260811 09:18:08.059470 11736 ts_tablet_manager.cc:1918] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:18:08.059640 11736 log.cc:1196] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/50a9790621df4e42b7adfb22c251815c
I20260811 09:18:08.060173 11736 ts_tablet_manager.cc:1939] T 50a9790621df4e42b7adfb22c251815c P b92630152cec469ebc7118282dfbeee2: Deleting consensus metadata
I20260811 09:18:08.061913 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:08.023676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37942 (local address 127.17.16.65:42121)
0811 09:18:08.023855 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:08.023862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:08.032076 (+ 8214us) server_negotiation.cc:407] Connection header received
0811 09:18:08.032267 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:08.032273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:08.032388 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:08.032488 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:08.034947 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:08.036231 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:08.038248 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:08.038969 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:08.039123 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:08.053970 (+ 14847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:08.053985 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:08.054002 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:08.054060 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:08.057841 (+ 3781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:08.058736 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:08.058745 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:08.058749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:08.058927 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:08.059373 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:08.059379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:08.059382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:08.059799 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:08.060094 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:08.061304 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0811 09:18:08.061540 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:18:08.061995 10891 catalog_manager.cc:4841] TS b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405): tablet 50a9790621df4e42b7adfb22c251815c (table test-table [id=2e090742a5ba4375a6b3b1bfaa2f766d]) successfully deleted
I20260811 09:18:08.062408 11735 ts_tablet_manager.cc:1939] T 50a9790621df4e42b7adfb22c251815c P 2dec3e1860984249950375bf6012fbdd: Deleting consensus metadata
I20260811 09:18:08.062651 11230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:08.158516 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 11023
I20260811 09:18:08.158646 10892 catalog_manager.cc:4841] TS 2dec3e1860984249950375bf6012fbdd (127.17.16.65:42121): tablet 50a9790621df4e42b7adfb22c251815c (table test-table [id=2e090742a5ba4375a6b3b1bfaa2f766d]) successfully deleted
I20260811 09:18:08.182992 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 11246
Aug 11 09:18:08 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:18:08.197369 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:08.183729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36218 (local address 127.17.16.66:34405)
0811 09:18:08.183885 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:08.183889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:08.183907 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:18:08.184060 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:08.184064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:08.184147 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:08.184231 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:08.185602 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:08.186307 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:08.187501 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:08.187880 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:08.187967 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:08.192091 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:08.192106 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:08.192121 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:08.192167 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:08.194820 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:08.195420 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:08.195428 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:08.195430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:08.195568 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:08.195907 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:08.195910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:08.195912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:08.196275 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:08.196487 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:08.196786 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 09:18:08.197081 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:18:08.198004 11450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:08.286897 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 11246
I20260811 09:18:08.310374 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 11464
Aug 11 09:18:08 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:18:08.325281 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:08.311143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33514 (local address 127.17.16.67:34061)
0811 09:18:08.311307 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:08.311312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:08.311330 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:18:08.311465 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:08.311469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:08.311530 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:08.311622 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:08.312828 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:08.313582 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:08.314905 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:08.315275 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:08.315359 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:08.319800 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:08.319817 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:08.319833 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:08.319866 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:08.322616 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:08.323270 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:08.323277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:08.323280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:08.323417 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:08.323790 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:08.323794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:08.323796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:08.324150 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:08.324434 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:08.324730 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 09:18:08.324959 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:18:08.325874 11665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:08.412670 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 11464
I20260811 09:18:08.434146 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 10867
I20260811 09:18:08.435272 10974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:08.543630 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 10867
I20260811 09:18:28.573632 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:35519
--webserver_interface=127.17.16.126
--webserver_port=41993
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:35519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:18:28.919751 11754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:28.920051 11754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:28.920190 11754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:28.927335 11754 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 09:18:28.927457 11754 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 09:18:28.927526 11754 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 09:18:28.927598 11754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:18:28.927672 11754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:28.927772 11754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:28.927832 11754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:18:28.927906 11754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:18:28.937698 11754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:35519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:35519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=41993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:28.938992 11754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:28.940310 11754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:28.964591 11762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:28.964564 11766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:28.964550 11770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:28.965342 11754 server_base.cc:1034] running on GCE node
I20260811 09:18:28.966208 11754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:28.967657 11754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:28.968884 11754 hybrid_clock.cc:648] HybridClock initialized: now 1786439908968812 us; error 43 us; skew 500 ppm
I20260811 09:18:28.969295 11754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:28 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439908, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:28.972729 11754 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:18:28.974314 11754 webserver.cc:460] Webserver started at http://127.17.16.126:41993/ using document root <none> and password file <none>
I20260811 09:18:28.975061 11754 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:28.975169 11754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:28.981371 11754 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 09:18:28.984200 11778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:28.985525 11754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:18:28.985678 11754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6"
format_stamp: "Formatted at 2026-08-11 09:18:05 on dist-test-slave-ncp9"
server_key: "1cff16cfcf17a05d1da46cfcae5921f3"
server_key_iv: "50d31734667d2cc9106fbe449ba29108"
server_key_version: "encryptionkey@0"
I20260811 09:18:28.986685 11754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:29.000049 11754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:29.004020 11754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:29.004262 11754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:29.022300 11754 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:35519
I20260811 09:18:29.022321 11857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:35519 every 8 connection(s)
I20260811 09:18:29.023808 11754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:18:29.026225 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 11754
I20260811 09:18:29.027271 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:42121
--local_ip_for_outbound_sockets=127.17.16.65
--tserver_master_addrs=127.17.16.126:35519
--webserver_port=35169
--webserver_interface=127.17.16.65
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:18:29.033094 11859 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:18:29.037851 11859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Bootstrap starting.
I20260811 09:18:29.059487 11859 log.cc:824] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Log is configured to *not* fsync() on all Append() calls
I20260811 09:18:29.073233 11859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:18:29.073853 11859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Bootstrap complete.
I20260811 09:18:29.083912 11859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:29.084829 11859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ffe6e12e0de4c74b3cbb39027e7b5e6, State: Initialized, Role: FOLLOWER
I20260811 09:18:29.085573 11859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:29.085831 11859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:18:29.085980 11859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:18:29.086160 11859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:18:29.090400 11859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:29.091030 11859 leader_election.cc:302] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ffe6e12e0de4c74b3cbb39027e7b5e6; no voters:
I20260811 09:18:29.091707 11859 leader_election.cc:288] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:18:29.092002 11872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:18:29.093031 11872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 2 LEADER]: Becoming Leader. State: Replica: 2ffe6e12e0de4c74b3cbb39027e7b5e6, State: Running, Role: LEADER
I20260811 09:18:29.093719 11872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:29.094115 11859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:18:29.096618 11875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ffe6e12e0de4c74b3cbb39027e7b5e6. Latest consensus state: current_term: 2 leader_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } } }
I20260811 09:18:29.097025 11875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: This master's current role is: LEADER
I20260811 09:18:29.097543 11873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } } }
I20260811 09:18:29.097837 11873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: This master's current role is: LEADER
I20260811 09:18:29.106241 11878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:18:29.115568 11878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:18:29.117280 11878 catalog_manager.cc:1259] Loaded cluster ID: 018d493d4e30478d81d4042c57d09195
I20260811 09:18:29.117472 11878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:18:29.122082 11878 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:18:29.124083 11878 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Loaded TSK: 0
I20260811 09:18:29.161626 11878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Generated new TSK 1
I20260811 09:18:29.162106 11878 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:18:29.403429 11867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:29.403716 11867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:29.403851 11867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:29.411020 11867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:29.411134 11867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:29.411285 11867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:18:29.421007 11867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:42121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=35169
--tserver_master_addrs=127.17.16.126:35519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:29.422313 11867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:29.423640 11867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:29.437602 11910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:29.437819 11915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:29.438253 11909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:29.439201 11867 server_base.cc:1034] running on GCE node
I20260811 09:18:29.439774 11867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:29.440716 11867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:29.441934 11867 hybrid_clock.cc:648] HybridClock initialized: now 1786439909441889 us; error 38 us; skew 500 ppm
I20260811 09:18:29.442265 11867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:29 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439909, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:29.446034 11867 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:18:29.448364 11867 webserver.cc:460] Webserver started at http://127.17.16.65:35169/ using document root <none> and password file <none>
I20260811 09:18:29.449357 11867 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:29.449564 11867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:29.455365 11867 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 09:18:29.458041 11925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:29.459563 11867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:18:29.459740 11867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "2dec3e1860984249950375bf6012fbdd"
format_stamp: "Formatted at 2026-08-11 09:18:06 on dist-test-slave-ncp9"
server_key: "b1419bda2b7fd87b54f9526a81d7afdd"
server_key_iv: "673877bed89e8caaff90bcf8f107a21d"
server_key_version: "encryptionkey@0"
I20260811 09:18:29.461159 11867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:29.495128 11867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:29.499578 11867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:29.499850 11867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:29.500972 11867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:18:29.502496 11867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:18:29.502593 11867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:29.502703 11867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:18:29.502774 11867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:29.533281 11867 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:42121
I20260811 09:18:29.533360 12099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:42121 every 8 connection(s)
I20260811 09:18:29.534770 11867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:18:29.540345 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 11867
I20260811 09:18:29.541097 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:34405
--local_ip_for_outbound_sockets=127.17.16.66
--tserver_master_addrs=127.17.16.126:35519
--webserver_port=44375
--webserver_interface=127.17.16.66
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:18:29 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439909, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:18:29.559024 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:29.537688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:57017 (local address 127.17.16.126:35519)
0811 09:18:29.538295 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:29.538306 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:29.538942 (+ 636us) server_negotiation.cc:407] Connection header received
0811 09:18:29.541285 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:29.541306 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:29.541733 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:29.542151 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:29.543520 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:29.545272 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:29.546770 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:29.547716 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:29.547889 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:29.551349 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:29.551359 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:29.551378 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:29.551423 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:29.554641 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:29.555517 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:29.555523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:29.555529 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:29.555699 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:29.556149 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:29.556152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:29.556154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:29.556612 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:29.556751 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:29.557758 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0811 09:18:29.558091 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":262,"threads_started":1}
I20260811 09:18:29.559337 12106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:29.537867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:57017)
0811 09:18:29.538766 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:29.538822 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:29.540612 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:29.542259 (+ 1647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:29.542267 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:29.542773 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:18:29.543318 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:29.543337 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:29.545419 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:29.545423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:29.546617 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:29.546630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:29.547903 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:29.547908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:29.548094 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:29.548715 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:18:29.548738 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:18:29.551050 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:18:29.554795 (+ 3745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:29.554804 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:29.554821 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:29.555406 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:29.555816 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:29.555821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:29.555825 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:29.556062 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:29.556767 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:18:29.556774 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:18:29.557245 (+ 471us) client_negotiation.cc:769] Sending connection context
0811 09:18:29.557569 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 09:18:29.558095 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":368,"threads_started":1}
I20260811 09:18:29.562350 12102 heartbeater.cc:347] Connected to a master server at 127.17.16.126:35519
I20260811 09:18:29.562727 12102 heartbeater.cc:464] Registering TS with master...
I20260811 09:18:29.563578 12102 heartbeater.cc:511] Master 127.17.16.126:35519 requested a full tablet report, sending...
I20260811 09:18:29.565981 11803 ts_manager.cc:194] Registered new tserver with Master: 2dec3e1860984249950375bf6012fbdd (127.17.16.65:42121)
I20260811 09:18:29.567579 11803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:57017
W20260811 09:18:29.884502 12107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:29.884763 12107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:29.884935 12107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:29.892032 12107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:29.892135 12107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:29.892285 12107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:18:29.901934 12107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:34405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=44375
--tserver_master_addrs=127.17.16.126:35519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:29.903128 12107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:29.904424 12107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:29.916779 12113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:29.916817 12114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:29.916922 12118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:29.917860 12107 server_base.cc:1034] running on GCE node
I20260811 09:18:29.918407 12107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:29.919258 12107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:29.920450 12107 hybrid_clock.cc:648] HybridClock initialized: now 1786439909920405 us; error 35 us; skew 500 ppm
I20260811 09:18:29.920729 12107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:29 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439909, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:29.923646 12107 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:18:29.925045 12107 webserver.cc:460] Webserver started at http://127.17.16.66:44375/ using document root <none> and password file <none>
I20260811 09:18:29.925698 12107 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:29.925825 12107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:29.930785 12107 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 09:18:29.933274 12126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:29.934481 12107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:18:29.934636 12107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "b92630152cec469ebc7118282dfbeee2"
format_stamp: "Formatted at 2026-08-11 09:18:06 on dist-test-slave-ncp9"
server_key: "50eb4e6a11ce4f8715677484065d1640"
server_key_iv: "a2ec63345a23b0bfe81a2337a36c8e64"
server_key_version: "encryptionkey@0"
I20260811 09:18:29.935563 12107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:29.957226 12107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:29.960834 12107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:29.961097 12107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:29.961992 12107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:18:29.963255 12107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:18:29.963356 12107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:29.963455 12107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:18:29.963532 12107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:29.993883 12107 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:34405
I20260811 09:18:29.993979 12291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:34405 every 8 connection(s)
I20260811 09:18:29.995396 12107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:18:30.005755 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 12107
I20260811 09:18:30.006390 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:34061
--local_ip_for_outbound_sockets=127.17.16.67
--tserver_master_addrs=127.17.16.126:35519
--webserver_port=32939
--webserver_interface=127.17.16.67
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:18:30 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439909, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:18:30.014581 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:29.997935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:48975 (local address 127.17.16.126:35519)
0811 09:18:29.998116 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:29.998121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:29.999233 (+ 1112us) server_negotiation.cc:407] Connection header received
0811 09:18:30.001075 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.001081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.001174 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.001263 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:30.002733 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.003454 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.004985 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.005410 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.005519 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.009242 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:30.009247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:30.009251 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:30.009283 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:30.011689 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.012470 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.012473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.012476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.012585 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.013035 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.013038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.013040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.013309 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:30.013418 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.014173 (+ 755us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.014337 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:18:30.015791 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:29.998379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:48975)
0811 09:18:29.999086 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:29.999124 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:30.000787 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:30.001505 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:30.001513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:30.002009 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:18:30.002556 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.002573 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.003600 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.003603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.004821 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.004834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.005645 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.005654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.005874 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.006488 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:18:30.006524 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:18:30.009017 (+ 2493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:18:30.011820 (+ 2803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:30.011828 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:30.011842 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:30.012371 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:30.012671 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:30.012675 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:30.012678 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:30.012940 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:30.013424 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:18:30.013431 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:18:30.013815 (+ 384us) client_negotiation.cc:769] Sending connection context
0811 09:18:30.014095 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 09:18:30.014574 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":280,"threads_started":1}
I20260811 09:18:30.017163 12292 heartbeater.cc:347] Connected to a master server at 127.17.16.126:35519
I20260811 09:18:30.017444 12292 heartbeater.cc:464] Registering TS with master...
I20260811 09:18:30.018124 12292 heartbeater.cc:511] Master 127.17.16.126:35519 requested a full tablet report, sending...
I20260811 09:18:30.019554 11803 ts_manager.cc:194] Registered new tserver with Master: b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
I20260811 09:18:30.020320 11803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:48975
W20260811 09:18:30.344065 12296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:30.344326 12296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:30.344468 12296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:30.351616 12296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:30.351727 12296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:30.351897 12296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:18:30.361420 12296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:34061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=32939
--tserver_master_addrs=127.17.16.126:35519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:30.362666 12296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:30.363906 12296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:30.377159 12308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:30.377225 12306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:30.377398 12305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:30.378108 12296 server_base.cc:1034] running on GCE node
I20260811 09:18:30.378711 12296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:30.379652 12296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:30.380847 12296 hybrid_clock.cc:648] HybridClock initialized: now 1786439910380798 us; error 39 us; skew 500 ppm
I20260811 09:18:30.381189 12296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:30 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439910, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:30.384186 12296 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:18:30.385615 12296 webserver.cc:460] Webserver started at http://127.17.16.67:32939/ using document root <none> and password file <none>
I20260811 09:18:30.386261 12296 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:30.386392 12296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:30.391659 12296 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 09:18:30.394243 12319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:30.395556 12296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:18:30.395720 12296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "b358b25eee3541609d8800dd14f09e48"
format_stamp: "Formatted at 2026-08-11 09:18:07 on dist-test-slave-ncp9"
server_key: "e94e72162ce38f02892dda0dab75b885"
server_key_iv: "7809d691fa7a183ccd680e7112a02e26"
server_key_version: "encryptionkey@0"
I20260811 09:18:30.396623 12296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:30.410343 12296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:30.414179 12296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:30.414395 12296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:30.415252 12296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:18:30.416433 12296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:18:30.416532 12296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:30.416610 12296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:18:30.416671 12296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:30.447325 12296 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:34061
I20260811 09:18:30.447410 12484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:34061 every 8 connection(s)
I20260811 09:18:30.448740 12296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:18:30.456605 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 12296
Aug 11 09:18:30 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439910, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:18:30.470265 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.451374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:40869 (local address 127.17.16.126:35519)
0811 09:18:30.451570 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.451574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.452618 (+ 1044us) server_negotiation.cc:407] Connection header received
0811 09:18:30.454512 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.454516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.454583 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.454671 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:30.456097 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.456933 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.458967 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.459680 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.459810 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.463948 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:30.463954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:30.463959 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:30.464004 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:30.466933 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.467758 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.467763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.467766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.467905 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.468385 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.468390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.468392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.468695 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:30.468843 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.469588 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.469838 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":67}
I20260811 09:18:30.471184 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.451725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:40869)
0811 09:18:30.452439 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:30.452485 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:30.454264 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:30.454906 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:30.454914 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:30.455408 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:18:30.455955 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.455972 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.457110 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.457117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.458811 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.458827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.459893 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.459900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.460141 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.460860 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:18:30.460914 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:18:30.463742 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:18:30.467094 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:30.467104 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:30.467123 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:30.467628 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:30.468031 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:30.468037 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:30.468042 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:30.468255 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:30.468829 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:18:30.468837 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:18:30.469190 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 09:18:30.469467 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 09:18:30.469999 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":264,"threads_started":1}
I20260811 09:18:30.472622 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.457721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42834 (local address 127.17.16.126:35519)
0811 09:18:30.458236 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.458243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.458573 (+ 330us) server_negotiation.cc:407] Connection header received
0811 09:18:30.458724 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.458730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.458810 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.458937 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:30.460849 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.462077 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.463743 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.464416 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.464581 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.467323 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:30.467327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:30.467331 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:30.467360 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:30.469879 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.470635 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.470640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.470644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.470757 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.471101 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.471106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.471110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.471399 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:30.471657 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.472008 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.472223 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":255,"threads_started":1}
I20260811 09:18:30.472739 12485 heartbeater.cc:347] Connected to a master server at 127.17.16.126:35519
I20260811 09:18:30.473095 12485 heartbeater.cc:464] Registering TS with master...
I20260811 09:18:30.474010 12485 heartbeater.cc:511] Master 127.17.16.126:35519 requested a full tablet report, sending...
I20260811 09:18:30.475589 11803 ts_manager.cc:194] Registered new tserver with Master: b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061)
I20260811 09:18:30.476415 11803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:40869
I20260811 09:18:30.487341 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:18:30.493574 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.488249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42840 (local address 127.17.16.126:35519)
0811 09:18:30.488407 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.488413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.488822 (+ 409us) server_negotiation.cc:407] Connection header received
0811 09:18:30.489036 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.489040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.489157 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.489275 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:18:30.489629 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.490319 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.491882 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.492277 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.492365 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.492768 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:18:30.493079 (+ 311us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 09:18:30.493351 (+ 272us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42840: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 09:18:30.493693 12490 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42840: token expired
W20260811 09:18:30.494292 11679 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.16.126:35519)
I20260811 09:18:30.506294 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.494901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42842 (local address 127.17.16.126:35519)
0811 09:18:30.495032 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.495036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.495050 (+ 14us) server_negotiation.cc:407] Connection header received
0811 09:18:30.495092 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.495095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.495143 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.495210 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:30.496488 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.497248 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.498440 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.498856 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.498947 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.501521 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:30.501527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:30.501530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:30.501562 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:30.504003 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.504572 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.504576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.504578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.504715 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.505066 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.505070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.505072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.505390 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:30.505588 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.505824 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.505986 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 09:18:30.509443 11803 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:18:30.516515 11803 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:18:30.538738 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.527894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.0.0.1:58412)
0811 09:18:30.528755 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:30.528795 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:30.529013 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:30.530303 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:30.530311 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:30.530344 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:18:30.530753 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.530773 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.532993 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.532998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.535478 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.535493 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.537574 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.537581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.537798 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.537855 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 09:18:30.537977 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 09:18:30.538163 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":124,"thread_start_us":575,"threads_started":1}
I20260811 09:18:30.538761 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.527899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:34061 (local address 127.0.0.1:56706)
0811 09:18:30.529952 (+ 2053us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:30.529973 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:30.530073 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:30.531222 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:30.531226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:30.531249 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:18:30.531625 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.531638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.533820 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.533826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.536025 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.536040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.537893 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.537899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.538063 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.538095 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:18:30.538182 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 09:18:30.538313 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1937,"mutex_wait_us":209,"thread_start_us":1586,"threads_started":1}
I20260811 09:18:30.539896 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.527908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58412 (local address 127.17.16.65:42121)
0811 09:18:30.529647 (+ 1739us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.529657 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.529686 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:18:30.529789 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.529798 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.530060 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.530991 (+ 931us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:18:30.531270 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.532823 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.535722 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.537311 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.538991 (+ 1680us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.539223 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.539296 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.539515 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1563,"thread_start_us":1419,"threads_started":1}
I20260811 09:18:30.540483 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.528293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.17.16.67:34061)
0811 09:18:30.530643 (+ 2350us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.530651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.530684 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:18:30.530777 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.530784 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.531001 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.531801 (+ 800us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:18:30.532022 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.533650 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.536226 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.537619 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.539554 (+ 1935us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.539759 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.539823 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.540180 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2186,"thread_start_us":2025,"threads_started":1}
I20260811 09:18:30.541687 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.530050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:34405 (local address 127.0.0.1:41444)
0811 09:18:30.530948 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:30.530963 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:30.531059 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:30.532062 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:30.532067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:30.532092 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:18:30.532447 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.532461 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.535447 (+ 2986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.535452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.538573 (+ 3121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.538587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.540783 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.540790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.540979 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.541011 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:18:30.541076 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 09:18:30.541189 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":694,"threads_started":1}
I20260811 09:18:30.542330 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.530530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41444 (local address 127.17.16.66:34405)
0811 09:18:30.531517 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.531537 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.531563 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:18:30.531664 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.531672 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.531875 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.532049 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:18:30.532649 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.535300 (+ 2651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.538811 (+ 3511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.540524 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.541597 (+ 1073us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.541783 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.541850 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.541995 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":1068,"threads_started":1}
I20260811 09:18:30.543530 12002 tablet_service.cc:1467] Processing CreateTablet for tablet a1cd60ac63044d818106e61a4ff3d33c (DEFAULT_TABLE table=test-table [id=04cbd9af6cea49f18b781ae31369b416]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:18:30.543836 12389 tablet_service.cc:1467] Processing CreateTablet for tablet a1cd60ac63044d818106e61a4ff3d33c (DEFAULT_TABLE table=test-table [id=04cbd9af6cea49f18b781ae31369b416]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:18:30.544982 12389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1cd60ac63044d818106e61a4ff3d33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:18:30.545038 12002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1cd60ac63044d818106e61a4ff3d33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:18:30.546224 12200 tablet_service.cc:1467] Processing CreateTablet for tablet a1cd60ac63044d818106e61a4ff3d33c (DEFAULT_TABLE table=test-table [id=04cbd9af6cea49f18b781ae31369b416]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:18:30.547528 12200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1cd60ac63044d818106e61a4ff3d33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:18:30.556695 12515 tablet_bootstrap.cc:492] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: Bootstrap starting.
I20260811 09:18:30.559980 12515 tablet_bootstrap.cc:654] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: Neither blocks nor log segments found. Creating new log.
I20260811 09:18:30.559979 12517 tablet_bootstrap.cc:492] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Bootstrap starting.
I20260811 09:18:30.562160 12515 log.cc:824] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: Log is configured to *not* fsync() on all Append() calls
I20260811 09:18:30.563683 12517 tablet_bootstrap.cc:654] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Neither blocks nor log segments found. Creating new log.
I20260811 09:18:30.565017 12517 log.cc:824] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Log is configured to *not* fsync() on all Append() calls
I20260811 09:18:30.565886 12519 tablet_bootstrap.cc:492] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: Bootstrap starting.
I20260811 09:18:30.567807 12515 tablet_bootstrap.cc:492] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: No bootstrap required, opened a new log
I20260811 09:18:30.568089 12515 ts_tablet_manager.cc:1397] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260811 09:18:30.568450 12517 tablet_bootstrap.cc:492] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: No bootstrap required, opened a new log
I20260811 09:18:30.568687 12517 ts_tablet_manager.cc:1397] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 09:18:30.570206 12519 tablet_bootstrap.cc:654] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: Neither blocks nor log segments found. Creating new log.
I20260811 09:18:30.571290 12102 heartbeater.cc:503] Master 127.17.16.126:35519 was elected leader, sending a full tablet report...
I20260811 09:18:30.572759 12519 log.cc:824] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:18:30.577925 12517 raft_consensus.cc:348] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:30.578256 12517 raft_consensus.cc:374] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:18:30.578353 12517 raft_consensus.cc:729] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dec3e1860984249950375bf6012fbdd, State: Initialized, Role: FOLLOWER
I20260811 09:18:30.578876 12517 consensus_queue.cc:260] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:30.579416 12519 tablet_bootstrap.cc:492] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: No bootstrap required, opened a new log
I20260811 09:18:30.579675 12519 ts_tablet_manager.cc:1397] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260811 09:18:30.582130 12515 raft_consensus.cc:348] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:30.582553 12515 raft_consensus.cc:374] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:18:30.582665 12515 raft_consensus.cc:729] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b358b25eee3541609d8800dd14f09e48, State: Initialized, Role: FOLLOWER
I20260811 09:18:30.583251 12515 consensus_queue.cc:260] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:30.586047 12485 heartbeater.cc:503] Master 127.17.16.126:35519 was elected leader, sending a full tablet report...
I20260811 09:18:30.586292 12517 ts_tablet_manager.cc:1428] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Time spent starting tablet: real 0.018s user 0.011s sys 0.007s
I20260811 09:18:30.586745 12515 ts_tablet_manager.cc:1428] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: Time spent starting tablet: real 0.019s user 0.012s sys 0.003s
I20260811 09:18:30.591089 12519 raft_consensus.cc:348] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:30.591405 12519 raft_consensus.cc:374] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:18:30.591499 12519 raft_consensus.cc:729] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92630152cec469ebc7118282dfbeee2, State: Initialized, Role: FOLLOWER
I20260811 09:18:30.591990 12519 consensus_queue.cc:260] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:30.592926 12292 heartbeater.cc:503] Master 127.17.16.126:35519 was elected leader, sending a full tablet report...
I20260811 09:18:30.593472 12519 ts_tablet_manager.cc:1428] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260811 09:18:30.700354 12486 tablet.cc:2369] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:18:30.714880 12531 raft_consensus.cc:482] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:18:30.715173 12531 raft_consensus.cc:504] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:30.716886 12531 leader_election.cc:288] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2dec3e1860984249950375bf6012fbdd (127.17.16.65:42121), b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
I20260811 09:18:30.723309 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.717267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:34405 (local address 127.17.16.67:52437)
0811 09:18:30.717571 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:30.717591 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:30.717712 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:30.718072 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:30.718077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:30.718109 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:18:30.718316 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.718327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.719500 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.719506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.721377 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.721393 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.722784 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.722788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.722920 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.722945 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:18:30.723014 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 09:18:30.723105 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260811 09:18:30.723441 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.717508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:52437 (local address 127.17.16.66:34405)
0811 09:18:30.717718 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.717724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.717747 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:18:30.717835 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.717841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.717921 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.718022 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:18:30.718450 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.719360 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.721559 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.722633 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.722800 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.722875 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.723062 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.723193 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:18:30.724123 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.717353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:58977 (local address 127.17.16.65:42121)
0811 09:18:30.717609 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.717615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.718263 (+ 648us) server_negotiation.cc:407] Connection header received
0811 09:18:30.718422 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.718427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.718507 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.718587 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:18:30.719160 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.719949 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.722078 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.723259 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.723419 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.723489 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.723814 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.723938 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 09:18:30.724162 12536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.717438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.17.16.67:58977)
0811 09:18:30.718141 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:30.718161 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:30.718277 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:30.718636 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:30.718640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:30.718667 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:18:30.718986 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.718998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.720084 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.720090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.721910 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:30.721921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.723462 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.723468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:30.723622 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.723664 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 09:18:30.723748 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 09:18:30.723860 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":422,"threads_started":1}
I20260811 09:18:30.724756 12225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1cd60ac63044d818106e61a4ff3d33c" candidate_uuid: "b358b25eee3541609d8800dd14f09e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92630152cec469ebc7118282dfbeee2" is_pre_election: true
I20260811 09:18:30.725296 12225 raft_consensus.cc:2454] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b358b25eee3541609d8800dd14f09e48 in term 0.
I20260811 09:18:30.725188 12033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1cd60ac63044d818106e61a4ff3d33c" candidate_uuid: "b358b25eee3541609d8800dd14f09e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dec3e1860984249950375bf6012fbdd" is_pre_election: true
I20260811 09:18:30.725682 12033 raft_consensus.cc:2454] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b358b25eee3541609d8800dd14f09e48 in term 0.
I20260811 09:18:30.725994 12320 leader_election.cc:302] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b358b25eee3541609d8800dd14f09e48, b92630152cec469ebc7118282dfbeee2; no voters:
I20260811 09:18:30.726393 12531 raft_consensus.cc:2781] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:18:30.726550 12531 raft_consensus.cc:482] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:18:30.726646 12531 raft_consensus.cc:3037] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:18:30.728755 12531 raft_consensus.cc:504] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:30.729373 12531 leader_election.cc:288] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [CANDIDATE]: Term 1 election: Requested vote from peers 2dec3e1860984249950375bf6012fbdd (127.17.16.65:42121), b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
I20260811 09:18:30.729810 12033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1cd60ac63044d818106e61a4ff3d33c" candidate_uuid: "b358b25eee3541609d8800dd14f09e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dec3e1860984249950375bf6012fbdd"
I20260811 09:18:30.729887 12225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1cd60ac63044d818106e61a4ff3d33c" candidate_uuid: "b358b25eee3541609d8800dd14f09e48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92630152cec469ebc7118282dfbeee2"
I20260811 09:18:30.729977 12033 raft_consensus.cc:3037] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:18:30.730082 12225 raft_consensus.cc:3037] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:18:30.732107 12033 raft_consensus.cc:2454] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b358b25eee3541609d8800dd14f09e48 in term 1.
I20260811 09:18:30.732296 12225 raft_consensus.cc:2454] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b358b25eee3541609d8800dd14f09e48 in term 1.
I20260811 09:18:30.732473 12321 leader_election.cc:302] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dec3e1860984249950375bf6012fbdd, b358b25eee3541609d8800dd14f09e48; no voters:
I20260811 09:18:30.732787 12531 raft_consensus.cc:2781] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:18:30.733242 12531 raft_consensus.cc:686] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 1 LEADER]: Becoming Leader. State: Replica: b358b25eee3541609d8800dd14f09e48, State: Running, Role: LEADER
I20260811 09:18:30.733662 12531 consensus_queue.cc:237] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:30.737864 11801 catalog_manager.cc:5495] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 reported cstate change: term changed from 0 to 1, leader changed from <none> to b358b25eee3541609d8800dd14f09e48 (127.17.16.67). New cstate: current_term: 1 leader_uuid: "b358b25eee3541609d8800dd14f09e48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } health_report { overall_health: UNKNOWN } } }
W20260811 09:18:30.747323 12293 tablet.cc:2369] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:18:30.776830 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.771818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56710 (local address 127.17.16.67:34061)
0811 09:18:30.771993 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.771998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.772016 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:18:30.772167 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.772172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.772287 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.772446 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:18:30.772717 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.773499 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.774925 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.775334 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.775422 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.775867 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:18:30.776362 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:18:30.776486 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.776523 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.776589 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:18:30.784584 12225 raft_consensus.cc:1261] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2 [term 1 FOLLOWER]: Refusing update from remote peer b358b25eee3541609d8800dd14f09e48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:18:30.784593 12033 raft_consensus.cc:1261] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Refusing update from remote peer b358b25eee3541609d8800dd14f09e48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:18:30.785521 12537 consensus_queue.cc:1035] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:18:30.785849 12531 consensus_queue.cc:1035] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 09:18:30.786973 12103 tablet.cc:2369] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:18:30.796744 12542 mvcc.cc:205] Tried to move back new op lower bound from 7317257874564276224 to 7317257874368577536. Current Snapshot: MvccSnapshot[applied={T|T < 7317257874564276224}]
I20260811 09:18:30.810819 11803 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:18:30.811065 11803 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:18:30.814461 11803 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Sending DeleteTablet for 3 replicas of tablet a1cd60ac63044d818106e61a4ff3d33c
I20260811 09:18:30.815663 12002 tablet_service.cc:1514] Processing DeleteTablet for tablet a1cd60ac63044d818106e61a4ff3d33c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:18:30 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:58412
I20260811 09:18:30.815683 12389 tablet_service.cc:1514] Processing DeleteTablet for tablet a1cd60ac63044d818106e61a4ff3d33c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:18:30 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:56706
I20260811 09:18:30.815845 12200 tablet_service.cc:1514] Processing DeleteTablet for tablet a1cd60ac63044d818106e61a4ff3d33c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:18:30 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:41444
I20260811 09:18:30.817032 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 11867
I20260811 09:18:30.831928 12558 tablet_replica.cc:330] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: stopping tablet replica
I20260811 09:18:30.833771 12558 raft_consensus.cc:2229] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:18:30.835166 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.818069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.17.16.65:42121)
0811 09:18:30.818262 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.818268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.818291 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:18:30.818485 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.818491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.818583 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.818673 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:30.820429 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.821646 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.823585 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.824250 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.824365 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.828002 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:30.828022 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:30.828043 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:30.828088 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:30.831941 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.832872 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.832881 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.832884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.833079 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.833475 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.833479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.833482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.834000 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:30.834308 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.834561 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.834809 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:18:30.835768 12095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:30.836023 12558 raft_consensus.cc:2258] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:18:30.836217 12560 tablet_replica.cc:330] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: stopping tablet replica
I20260811 09:18:30.836643 12560 raft_consensus.cc:2229] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:18:30.837198 12560 raft_consensus.cc:2258] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:18:30.838529 12558 ts_tablet_manager.cc:1905] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:18:30.843509 12560 ts_tablet_manager.cc:1905] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:18:30.846266 12558 ts_tablet_manager.cc:1918] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:18:30.846450 12558 log.cc:1196] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/a1cd60ac63044d818106e61a4ff3d33c
I20260811 09:18:30.847013 12558 ts_tablet_manager.cc:1939] T a1cd60ac63044d818106e61a4ff3d33c P b358b25eee3541609d8800dd14f09e48: Deleting consensus metadata
I20260811 09:18:30.848870 11779 catalog_manager.cc:4841] TS b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061): tablet a1cd60ac63044d818106e61a4ff3d33c (table test-table [id=04cbd9af6cea49f18b781ae31369b416]) successfully deleted
I20260811 09:18:30.855290 12560 ts_tablet_manager.cc:1918] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:18:30.855448 12560 log.cc:1196] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/a1cd60ac63044d818106e61a4ff3d33c
I20260811 09:18:30.856031 12560 ts_tablet_manager.cc:1939] T a1cd60ac63044d818106e61a4ff3d33c P b92630152cec469ebc7118282dfbeee2: Deleting consensus metadata
I20260811 09:18:30.857793 11779 catalog_manager.cc:4841] TS b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405): tablet a1cd60ac63044d818106e61a4ff3d33c (table test-table [id=04cbd9af6cea49f18b781ae31369b416]) successfully deleted
I20260811 09:18:30.939306 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 11867
I20260811 09:18:30.939488 12556 tablet_replica.cc:330] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: stopping tablet replica
W20260811 09:18:30.961995 11781 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.16.65:42121 (error 108)
I20260811 09:18:30.962316 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 12107
I20260811 09:18:30.964190 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.963636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.0.0.1:58440)
0811 09:18:30.963871 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:30.964008 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:42121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 09:18:30.964504 11781 catalog_manager.cc:4584] TS 2dec3e1860984249950375bf6012fbdd (127.17.16.65:42121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1cd60ac63044d818106e61a4ff3d33c: Network error: Client connection negotiation failed: client connection to 127.17.16.65:42121: connect: Connection refused (error 111)
I20260811 09:18:30.976140 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:30.963234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.17.16.66:34405)
0811 09:18:30.963412 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:30.963417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:30.963505 (+ 88us) server_negotiation.cc:407] Connection header received
0811 09:18:30.963709 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:30.963715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:30.963803 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:30.963909 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:30.965412 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.966175 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.967491 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:30.967886 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:30.967996 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:30.970666 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:30.970680 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:30.970695 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:30.970727 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:30.973442 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.974125 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.974132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.974135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.974255 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:30.974618 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:30.974622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:30.974624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:30.975036 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:30.975322 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:30.975618 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 09:18:30.975832 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:18:30.976931 12287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:31.018460 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:31.017905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.0.0.1:58454)
0811 09:18:31.018165 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:31.018287 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:42121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 09:18:31.019722 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:31.019252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.0.0.1:58466)
0811 09:18:31.019428 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:31.019539 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:42121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 09:18:31.080200 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:31.079773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.0.0.1:58472)
0811 09:18:31.079960 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:31.080057 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:42121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 09:18:31.081334 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:31.081003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.0.0.1:58486)
0811 09:18:31.081144 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:31.081221 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:42121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 09:18:31.085019 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 12107
I20260811 09:18:31.115512 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 12296
I20260811 09:18:31.129864 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:31.116262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56718 (local address 127.17.16.67:34061)
0811 09:18:31.116445 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:31.116451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:31.116569 (+ 118us) server_negotiation.cc:407] Connection header received
0811 09:18:31.116761 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:31.116765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:31.116830 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:31.116944 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:31.118312 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:31.119118 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:31.120531 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:31.121152 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:31.121267 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:31.123878 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:31.123889 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:31.123905 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:31.123941 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:31.127099 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:31.127749 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:31.127760 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:31.127762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:31.127916 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:31.128278 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:31.128283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:31.128286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:31.128653 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:31.128966 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:31.129278 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 09:18:31.129499 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:18:31.130501 12477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:31.161973 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:31.161513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.0.0.1:58490)
0811 09:18:31.161724 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:31.161813 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:42121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 09:18:31.163208 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:31.162637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.0.0.1:58504)
0811 09:18:31.162932 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:31.163046 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:42121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260811 09:18:31.251060 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 12296
I20260811 09:18:31.280025 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 11754
I20260811 09:18:31.281240 11850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:18:31.394081 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:31.393475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:42121 (local address 127.0.0.1:58510)
0811 09:18:31.393680 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:31.393791 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:42121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 09:18:31.394259 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 11754
I20260811 09:18:51.422350 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:35519
--webserver_interface=127.17.16.126
--webserver_port=41993
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:35519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:18:51.762557 12569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:51.762841 12569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:51.762948 12569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:51.770037 12569 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 09:18:51.770136 12569 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 09:18:51.770174 12569 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 09:18:51.770232 12569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:18:51.770285 12569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:51.770354 12569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:51.770397 12569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:18:51.770449 12569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:18:51.780517 12569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:35519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:35519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=41993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:51.781939 12569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:51.783305 12569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:51.795100 12576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:51.795154 12584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:51.795193 12578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:51.795396 12569 server_base.cc:1034] running on GCE node
I20260811 09:18:51.796432 12569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:51.797876 12569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:51.799098 12569 hybrid_clock.cc:648] HybridClock initialized: now 1786439931799026 us; error 49 us; skew 500 ppm
I20260811 09:18:51.799418 12569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:51 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439931, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:51.802580 12569 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:18:51.804049 12569 webserver.cc:460] Webserver started at http://127.17.16.126:41993/ using document root <none> and password file <none>
I20260811 09:18:51.804749 12569 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:51.804852 12569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:51.810217 12569 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 09:18:51.812930 12594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:51.814188 12569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:18:51.814344 12569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6"
format_stamp: "Formatted at 2026-08-11 09:18:05 on dist-test-slave-ncp9"
server_key: "1cff16cfcf17a05d1da46cfcae5921f3"
server_key_iv: "50d31734667d2cc9106fbe449ba29108"
server_key_version: "encryptionkey@0"
I20260811 09:18:51.815268 12569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:51.841184 12569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:51.845373 12569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:51.845667 12569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:51.863756 12569 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:35519
I20260811 09:18:51.863780 12677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:35519 every 8 connection(s)
I20260811 09:18:51.865223 12569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:18:51.872009 12678 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:18:51.873412 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 12569
I20260811 09:18:51.874310 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:42121
--local_ip_for_outbound_sockets=127.17.16.65
--tserver_master_addrs=127.17.16.126:35519
--webserver_port=35169
--webserver_interface=127.17.16.65
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:18:51.875581 12678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Bootstrap starting.
I20260811 09:18:51.895977 12678 log.cc:824] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Log is configured to *not* fsync() on all Append() calls
I20260811 09:18:51.912014 12678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:18:51.912555 12678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Bootstrap complete.
I20260811 09:18:51.921644 12678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:51.922580 12678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ffe6e12e0de4c74b3cbb39027e7b5e6, State: Initialized, Role: FOLLOWER
I20260811 09:18:51.923159 12678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:51.923384 12678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:18:51.923509 12678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:18:51.923666 12678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 2 FOLLOWER]: Advancing to term 3
I20260811 09:18:51.927456 12678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:51.927909 12678 leader_election.cc:302] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ffe6e12e0de4c74b3cbb39027e7b5e6; no voters:
I20260811 09:18:51.928560 12678 leader_election.cc:288] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 09:18:51.928707 12683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 09:18:51.929854 12683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [term 3 LEADER]: Becoming Leader. State: Replica: 2ffe6e12e0de4c74b3cbb39027e7b5e6, State: Running, Role: LEADER
I20260811 09:18:51.930450 12683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } }
I20260811 09:18:51.930747 12678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:18:51.933060 12688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } } }
I20260811 09:18:51.933446 12688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: This master's current role is: LEADER
I20260811 09:18:51.935119 12695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:18:51.934943 12692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ffe6e12e0de4c74b3cbb39027e7b5e6. Latest consensus state: current_term: 3 leader_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ffe6e12e0de4c74b3cbb39027e7b5e6" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 35519 } } }
I20260811 09:18:51.935384 12692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6 [sys.catalog]: This master's current role is: LEADER
I20260811 09:18:51.943773 12695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:18:51.945292 12695 catalog_manager.cc:1259] Loaded cluster ID: 018d493d4e30478d81d4042c57d09195
I20260811 09:18:51.945468 12695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:18:51.949576 12695 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:18:51.951511 12695 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Loaded TSK: 0
I20260811 09:18:51.951730 12695 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2ffe6e12e0de4c74b3cbb39027e7b5e6: Loaded TSK: 1
F20260811 09:18:51.952917 12695 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f37d6eb89ad google::LogMessage::Fail() at ??:0
@ 0x7f37d6ebb88a google::LogMessage::SendToLog() at ??:0
@ 0x7f37d6eb84de google::LogMessage::Flush() at ??:0
@ 0x7f37d6eb9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f37dac1c7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f37dac0b73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f37ec8e71c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f37ec8e6783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f37ec9722a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f37ec9720c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f37ec9bc5e9 std::function<>::operator()() at ??:0
@ 0x7f37ec8eacdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f37ec8e8c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f37ec970610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f37ec970439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f37d9b04345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f37d9b0791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f37d9b07799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f37d9ae2eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f37dcdbc6db start_thread at ??:0
@ 0x7f37d5d5261f clone at ??:0
W20260811 09:18:52.265172 12680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:52.265443 12680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:52.265581 12680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:52.272924 12680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:52.273039 12680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:52.273181 12680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:18:52.283139 12680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:42121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=35169
--tserver_master_addrs=127.17.16.126:35519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:52.284457 12680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:52.285878 12680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:52.298713 12765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:52.298731 12768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:52.298897 12766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:52.299759 12680 server_base.cc:1034] running on GCE node
I20260811 09:18:52.300312 12680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:52.301216 12680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:52.302450 12680 hybrid_clock.cc:648] HybridClock initialized: now 1786439932302388 us; error 48 us; skew 500 ppm
I20260811 09:18:52.302800 12680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:52 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439932, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:52.305801 12680 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:18:52.307228 12680 webserver.cc:460] Webserver started at http://127.17.16.65:35169/ using document root <none> and password file <none>
I20260811 09:18:52.307902 12680 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:52.308024 12680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:52.313280 12680 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:18:52.316099 12779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:52.317414 12680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:18:52.317567 12680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "2dec3e1860984249950375bf6012fbdd"
format_stamp: "Formatted at 2026-08-11 09:18:06 on dist-test-slave-ncp9"
server_key: "b1419bda2b7fd87b54f9526a81d7afdd"
server_key_iv: "673877bed89e8caaff90bcf8f107a21d"
server_key_version: "encryptionkey@0"
I20260811 09:18:52.318528 12680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:52.339210 12680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:52.343218 12680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:52.343463 12680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:52.344429 12680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:18:52.346315 12792 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 09:18:52.351754 12680 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 09:18:52.351859 12680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 09:18:52.351969 12680 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 09:18:52.355084 12680 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 09:18:52.355165 12680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260811 09:18:52.355625 12792 tablet_bootstrap.cc:492] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Bootstrap starting.
I20260811 09:18:52.383437 12792 log.cc:824] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Log is configured to *not* fsync() on all Append() calls
I20260811 09:18:52.394591 12792 tablet_bootstrap.cc:492] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:18:52.395143 12792 tablet_bootstrap.cc:492] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Bootstrap complete.
I20260811 09:18:52.396363 12792 ts_tablet_manager.cc:1397] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Time spent bootstrapping tablet: real 0.041s user 0.023s sys 0.013s
I20260811 09:18:52.404152 12792 raft_consensus.cc:348] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:52.405195 12792 raft_consensus.cc:729] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dec3e1860984249950375bf6012fbdd, State: Initialized, Role: FOLLOWER
I20260811 09:18:52.405974 12792 consensus_queue.cc:260] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:52.407423 12680 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:42121
I20260811 09:18:52.407503 12967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:42121 every 8 connection(s)
I20260811 09:18:52.408125 12792 ts_tablet_manager.cc:1428] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd: Time spent starting tablet: real 0.012s user 0.014s sys 0.000s
I20260811 09:18:52.408706 12680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:18:52.413367 12971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:52.411471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:45449)
0811 09:18:52.412225 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:52.412473 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":322,"threads_started":1}
W20260811 09:18:52.413933 12968 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:35519 (0 consecutive failures): Network error: Failed to ping master at 127.17.16.126:35519: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
I20260811 09:18:52.414701 12971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:52.414390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:55141)
0811 09:18:52.414534 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:52.414601 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 09:18:52.415454 12971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:52.415159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:57649)
0811 09:18:52.415272 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:52.415342 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 09:18:52.415654 12968 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:18:52.416865 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 12680
I20260811 09:18:52.417520 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:34405
--local_ip_for_outbound_sockets=127.17.16.66
--tserver_master_addrs=127.17.16.126:35519
--webserver_port=44375
--webserver_interface=127.17.16.66
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:18:52.752599 12975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:52.752869 12975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:52.753001 12975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:52.760110 12975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:52.760211 12975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:52.760327 12975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:18:52.770202 12975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:34405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=44375
--tserver_master_addrs=127.17.16.126:35519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:52.771409 12975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:52.772733 12975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:52.785976 12982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:52.785929 12983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:52.785889 12987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:52.786744 12975 server_base.cc:1034] running on GCE node
I20260811 09:18:52.787341 12975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:52.788244 12975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:52.789460 12975 hybrid_clock.cc:648] HybridClock initialized: now 1786439932789416 us; error 29 us; skew 500 ppm
I20260811 09:18:52.789778 12975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:52 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439932, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:52.792771 12975 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:18:52.794195 12975 webserver.cc:460] Webserver started at http://127.17.16.66:44375/ using document root <none> and password file <none>
I20260811 09:18:52.794840 12975 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:52.794942 12975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:52.800166 12975 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 09:18:52.802780 12995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:52.804071 12975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:18:52.804236 12975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "b92630152cec469ebc7118282dfbeee2"
format_stamp: "Formatted at 2026-08-11 09:18:06 on dist-test-slave-ncp9"
server_key: "50eb4e6a11ce4f8715677484065d1640"
server_key_iv: "a2ec63345a23b0bfe81a2337a36c8e64"
server_key_version: "encryptionkey@0"
I20260811 09:18:52.805274 12975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:52.823261 12975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:52.827458 12975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:52.827718 12975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:52.828886 12975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:18:52.831434 12975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:18:52.831544 12975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:52.831662 12975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:18:52.831738 12975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:52.861492 12975 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:34405
I20260811 09:18:52.861549 13179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:34405 every 8 connection(s)
I20260811 09:18:52.862963 12975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:18:52.867184 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 12975
I20260811 09:18:52.867758 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:52.865854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:52163)
0811 09:18:52.866624 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:52.866852 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":290,"threads_started":1}
I20260811 09:18:52.867800 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:34061
--local_ip_for_outbound_sockets=127.17.16.67
--tserver_master_addrs=127.17.16.126:35519
--webserver_port=32939
--webserver_interface=127.17.16.67
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:18:52.868480 13180 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:35519 (0 consecutive failures): Network error: Failed to ping master at 127.17.16.126:35519: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
I20260811 09:18:52.869356 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:52.869032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:49119)
0811 09:18:52.869175 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:52.869248 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 09:18:52.870200 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:52.869838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:57191)
0811 09:18:52.869975 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:52.870092 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 09:18:52.870411 13180 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:18:53.205469 13184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:18:53.205749 13184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:18:53.205899 13184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:18:53.213433 13184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:18:53.213546 13184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:18:53.213707 13184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:18:53.224115 13184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:34061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=32939
--tserver_master_addrs=127.17.16.126:35519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:18:53.225435 13184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:18:53.226732 13184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:18:53.239248 13193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:53.239281 13197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:18:53.239310 13202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:18:53.239952 13184 server_base.cc:1034] running on GCE node
I20260811 09:18:53.240562 13184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:18:53.241534 13184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:18:53.242753 13184 hybrid_clock.cc:648] HybridClock initialized: now 1786439933242695 us; error 45 us; skew 500 ppm
I20260811 09:18:53.243068 13184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:18:53 dist-test-slave-ncp9 krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439933, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:18:53.252379 13184 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:18:53.253913 13184 webserver.cc:460] Webserver started at http://127.17.16.67:32939/ using document root <none> and password file <none>
I20260811 09:18:53.254592 13184 fs_manager.cc:359] Metadata directory not provided
I20260811 09:18:53.254696 13184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:18:53.259992 13184 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 09:18:53.262620 13211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:53.263876 13184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:18:53.264015 13184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "b358b25eee3541609d8800dd14f09e48"
format_stamp: "Formatted at 2026-08-11 09:18:07 on dist-test-slave-ncp9"
server_key: "e94e72162ce38f02892dda0dab75b885"
server_key_iv: "7809d691fa7a183ccd680e7112a02e26"
server_key_version: "encryptionkey@0"
I20260811 09:18:53.265015 13184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:18:53.282169 13184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:18:53.286355 13184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:18:53.286604 13184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:18:53.287557 13184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:18:53.288875 13184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:18:53.288991 13184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:53.289070 13184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:18:53.289127 13184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:18:53.319680 13184 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:34061
I20260811 09:18:53.319747 13409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:34061 every 8 connection(s)
I20260811 09:18:53.321220 13184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:18:53.325954 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:53.324129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:36759)
0811 09:18:53.324868 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:53.325130 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":311,"threads_started":1}
W20260811 09:18:53.326553 13410 heartbeater.cc:644] Failed to heartbeat to 127.17.16.126:35519 (0 consecutive failures): Network error: Failed to ping master at 127.17.16.126:35519: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
I20260811 09:18:53.327298 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:53.326991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:51963)
0811 09:18:53.327118 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:53.327194 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 09:18:53.328028 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:53.327744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:45773)
0811 09:18:53.327849 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:53.327917 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 09:18:53.328233 13410 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:18:53.329493 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 13184
I20260811 09:18:53.416956 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:53.416191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:38521)
0811 09:18:53.416644 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:53.416730 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":205,"threads_started":1}
I20260811 09:18:53.871883 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:53.871035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:40203)
0811 09:18:53.871506 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:53.871649 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":224,"threads_started":1}
I20260811 09:18:54.245313 13418 raft_consensus.cc:482] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:18:54.245647 13418 raft_consensus.cc:504] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:54.247255 13418 leader_election.cc:288] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061), b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
Aug 11 09:18:54 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439932, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
Aug 11 09:18:54 dist-test-slave-ncp9 krb5kdc[10850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439932, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:18:54.267504 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:54.248629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:51633 (local address 127.17.16.66:34405)
0811 09:18:54.249363 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:54.249377 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:54.249419 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:18:54.250895 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:54.250903 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:54.251211 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:54.251487 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:54.253374 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:54.254509 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:54.256280 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:54.256925 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:54.257089 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:54.261173 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:54.261180 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:54.261191 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:54.261227 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:54.263988 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:54.264714 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:54.264720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:54.264725 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:54.264877 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:54.265314 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:54.265318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:54.265321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:54.265827 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:54.265989 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:54.266790 (+ 801us) server_negotiation.cc:299] Negotiation successful
0811 09:18:54.267126 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":438,"threads_started":1}
I20260811 09:18:54.267593 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:54.247770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:42075 (local address 127.17.16.67:34061)
0811 09:18:54.248371 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0811 09:18:54.248387 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:18:54.248429 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:18:54.250693 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:18:54.250703 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:18:54.251059 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:18:54.251367 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:18:54.253374 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:54.254523 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:54.256280 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:18:54.256928 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:18:54.257091 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:54.260853 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:18:54.260861 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:18:54.260875 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:18:54.260946 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:18:54.263881 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:54.264714 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:54.264722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:54.264730 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:54.264937 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:18:54.265313 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:18:54.265318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:18:54.265322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:18:54.265827 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:18:54.265989 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:18:54.266765 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 09:18:54.267136 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":285,"threads_started":1}
I20260811 09:18:54.267602 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:54.247474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:34061 (local address 127.17.16.65:42075)
0811 09:18:54.247909 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:54.247962 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:54.248569 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:54.251514 (+ 2945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:54.251525 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:54.252457 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:18:54.253192 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:54.253213 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:54.254700 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:54.254705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:54.256148 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:54.256161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:54.257133 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:54.257140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:54.257414 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:54.258178 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:18:54.258195 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:18:54.260579 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:18:54.264036 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:54.264046 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:54.264061 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:54.264589 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:54.265064 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:54.265069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:54.265073 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:54.265213 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:54.265978 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:18:54.265983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:18:54.266299 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 09:18:54.266542 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 09:18:54.267083 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":267,"spinlock_wait_cycles":17792,"thread_start_us":210,"threads_started":1}
I20260811 09:18:54.268148 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:54.248108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:34405 (local address 127.17.16.65:51633)
0811 09:18:54.248691 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:54.248709 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:18:54.248829 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:18:54.251515 (+ 2686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:18:54.251525 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:18:54.252455 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:18:54.253193 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:54.253213 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:54.254704 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:54.254708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:54.256149 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:18:54.256161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:18:54.257160 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:18:54.257168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:18:54.257408 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:18:54.258133 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:18:54.258155 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:18:54.260928 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:18:54.264121 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:54.264127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:54.264131 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:54.264589 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:54.265006 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:18:54.265012 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:18:54.265016 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:18:54.265208 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:18:54.265972 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:18:54.265979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:18:54.266300 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 09:18:54.266542 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 09:18:54.267244 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":409,"spinlock_wait_cycles":64000,"thread_start_us":638,"threads_started":1}
W20260811 09:18:54.271351 12780 leader_election.cc:341] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061): Not found: Tablet not found: a1cd60ac63044d818106e61a4ff3d33c
W20260811 09:18:54.271674 12780 leader_election.cc:341] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405): Not found: Tablet not found: a1cd60ac63044d818106e61a4ff3d33c
I20260811 09:18:54.271795 12780 leader_election.cc:302] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dec3e1860984249950375bf6012fbdd; no voters: b358b25eee3541609d8800dd14f09e48, b92630152cec469ebc7118282dfbeee2
I20260811 09:18:54.272184 13418 raft_consensus.cc:2726] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:18:54.329553 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:54.328713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:32995)
0811 09:18:54.329184 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:54.329320 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":240,"threads_started":1}
I20260811 09:18:54.418150 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:54.417776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:58673)
0811 09:18:54.417919 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:54.418006 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 09:18:54.873651 13431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:54.872860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:53687)
0811 09:18:54.873391 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:54.873482 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":278,"threads_started":1}
I20260811 09:18:55.330962 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:55.330328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:50023)
0811 09:18:55.330739 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:55.330815 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":211,"threads_started":1}
I20260811 09:18:55.419667 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:55.418946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:42687)
0811 09:18:55.419393 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:55.419481 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":236,"threads_started":1}
I20260811 09:18:55.875368 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:55.874512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:58783)
0811 09:18:55.875046 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:55.875158 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":303,"threads_started":1}
I20260811 09:18:55.877921 13436 raft_consensus.cc:482] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:18:55.878104 13436 raft_consensus.cc:504] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:55.878736 13436 leader_election.cc:288] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061), b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
W20260811 09:18:55.879743 12780 leader_election.cc:341] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405): Not found: Tablet not found: a1cd60ac63044d818106e61a4ff3d33c
W20260811 09:18:55.879961 12780 leader_election.cc:341] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061): Not found: Tablet not found: a1cd60ac63044d818106e61a4ff3d33c
I20260811 09:18:55.880105 12780 leader_election.cc:302] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dec3e1860984249950375bf6012fbdd; no voters: b358b25eee3541609d8800dd14f09e48, b92630152cec469ebc7118282dfbeee2
I20260811 09:18:55.880338 13436 raft_consensus.cc:2726] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:18:56.332448 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:56.331727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:58493)
0811 09:18:56.332196 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:56.332281 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":242,"threads_started":1}
I20260811 09:18:56.421345 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:56.420609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:35749)
0811 09:18:56.421082 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:56.421184 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":240,"threads_started":1}
I20260811 09:18:56.877152 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:56.876325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:48113)
0811 09:18:56.876827 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:56.876954 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":237,"threads_started":1}
I20260811 09:18:57.334019 13440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:57.333334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:40501)
0811 09:18:57.333760 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:57.333840 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":205,"threads_started":1}
I20260811 09:18:57.423079 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:57.422275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:33559)
0811 09:18:57.422804 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:57.422893 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":238,"threads_started":1}
I20260811 09:18:57.691885 13444 raft_consensus.cc:482] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:18:57.692111 13444 raft_consensus.cc:504] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:18:57.692734 13444 leader_election.cc:288] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061), b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
W20260811 09:18:57.693747 12780 leader_election.cc:341] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405): Not found: Tablet not found: a1cd60ac63044d818106e61a4ff3d33c
W20260811 09:18:57.693958 12780 leader_election.cc:341] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061): Not found: Tablet not found: a1cd60ac63044d818106e61a4ff3d33c
I20260811 09:18:57.694069 12780 leader_election.cc:302] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dec3e1860984249950375bf6012fbdd; no voters: b358b25eee3541609d8800dd14f09e48, b92630152cec469ebc7118282dfbeee2
I20260811 09:18:57.694319 13444 raft_consensus.cc:2726] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:18:57.878805 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:57.878077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:38139)
0811 09:18:57.878541 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:57.878631 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":233,"threads_started":1}
I20260811 09:18:58.335667 13446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:58.334923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:54991)
0811 09:18:58.335409 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:58.335493 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":286,"threads_started":1}
I20260811 09:18:58.424559 13447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:58.423894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:48805)
0811 09:18:58.424297 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:58.424381 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":210,"threads_started":1}
I20260811 09:18:58.880489 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:58.879694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:40147)
0811 09:18:58.880184 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:58.880270 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":251,"threads_started":1}
I20260811 09:18:59.337291 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:59.336545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:40957)
0811 09:18:59.337025 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:59.337112 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":234,"threads_started":1}
I20260811 09:18:59.426326 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:59.425566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:36479)
0811 09:18:59.426028 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:59.426123 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":229,"threads_started":1}
I20260811 09:18:59.882319 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:18:59.881388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:34445)
0811 09:18:59.881996 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 09:18:59.882096 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":281,"threads_started":1}
I20260811 09:19:00.338960 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:00.338190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:39359)
0811 09:19:00.338697 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:00.338779 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":220,"threads_started":1}
I20260811 09:19:00.427930 13458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:00.427210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:55275)
0811 09:19:00.427682 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:00.427766 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":247,"threads_started":1}
I20260811 09:19:00.884110 13459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:00.883260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:40975)
0811 09:19:00.883804 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:00.883915 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":309,"threads_started":1}
I20260811 09:19:01.340690 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:01.339882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:41669)
0811 09:19:01.340366 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:01.340470 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":256,"threads_started":1}
I20260811 09:19:01.429656 13462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:01.428876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:37929)
0811 09:19:01.429401 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:01.429486 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":281,"threads_started":1}
I20260811 09:19:01.885830 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:01.885112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:35847)
0811 09:19:01.885581 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:01.885668 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":216,"threads_started":1}
I20260811 09:19:02.248111 13465 raft_consensus.cc:482] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:19:02.248348 13465 raft_consensus.cc:504] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:19:02.249028 13465 leader_election.cc:288] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061), b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
W20260811 09:19:02.250013 12780 leader_election.cc:341] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061): Not found: Tablet not found: a1cd60ac63044d818106e61a4ff3d33c
W20260811 09:19:02.250265 12780 leader_election.cc:341] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405): Not found: Tablet not found: a1cd60ac63044d818106e61a4ff3d33c
I20260811 09:19:02.250356 12780 leader_election.cc:302] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dec3e1860984249950375bf6012fbdd; no voters: b358b25eee3541609d8800dd14f09e48, b92630152cec469ebc7118282dfbeee2
I20260811 09:19:02.250598 13465 raft_consensus.cc:2726] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:19:02.342350 13466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:02.341603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:41935)
0811 09:19:02.342094 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:02.342177 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":234,"threads_started":1}
I20260811 09:19:02.431385 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:02.430544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:48377)
0811 09:19:02.431043 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:02.431194 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":256,"threads_started":1}
I20260811 09:19:02.887398 13469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:02.886688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:40793)
0811 09:19:02.887123 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:02.887213 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":228,"threads_started":1}
I20260811 09:19:03.344102 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:03.343360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:56927)
0811 09:19:03.343834 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:03.343923 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":243,"threads_started":1}
I20260811 09:19:03.433038 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:03.432307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:44943)
0811 09:19:03.432739 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:03.432822 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":226,"threads_started":1}
I20260811 09:19:03.889091 13476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:03.888300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:55959)
0811 09:19:03.888784 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:03.888868 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":250,"threads_started":1}
I20260811 09:19:04.345841 13478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:04.345081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:57437)
0811 09:19:04.345596 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:04.345682 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":253,"threads_started":1}
I20260811 09:19:04.434645 13479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:04.433923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:46129)
0811 09:19:04.434387 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:04.434474 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":264,"threads_started":1}
I20260811 09:19:04.890825 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:04.890055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:38039)
0811 09:19:04.890541 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:04.890643 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":233,"threads_started":1}
I20260811 09:19:05.347496 13482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:05.346791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:47131)
0811 09:19:05.347243 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:05.347323 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":230,"threads_started":1}
I20260811 09:19:05.436223 13483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:05.435498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:34871)
0811 09:19:05.435946 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:05.436025 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":247,"threads_started":1}
I20260811 09:19:05.892473 13484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:05.891735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:42309)
0811 09:19:05.892210 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:05.892311 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":235,"threads_started":1}
I20260811 09:19:06.349066 13487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:06.348324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:44007)
0811 09:19:06.348779 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:06.348867 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":228,"threads_started":1}
I20260811 09:19:06.438022 13490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:06.437147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:51291)
0811 09:19:06.437659 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:06.437770 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":244,"threads_started":1}
I20260811 09:19:06.894033 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:06.893319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:34093)
0811 09:19:06.893755 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:06.893848 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":222,"threads_started":1}
I20260811 09:19:07.350682 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:07.349949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:34391)
0811 09:19:07.350397 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:07.350481 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":222,"threads_started":1}
I20260811 09:19:07.439842 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:07.439121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.65:45591)
0811 09:19:07.439552 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:07.439634 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":210,"threads_started":1}
I20260811 09:19:07.896014 13497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:07.895195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.66:37611)
0811 09:19:07.895742 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:07.895843 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":265,"threads_started":1}
I20260811 09:19:08.227972 13498 raft_consensus.cc:482] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:19:08.228173 13498 raft_consensus.cc:504] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dec3e1860984249950375bf6012fbdd" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 42121 } } peers { permanent_uuid: "b358b25eee3541609d8800dd14f09e48" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 34061 } } peers { permanent_uuid: "b92630152cec469ebc7118282dfbeee2" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 34405 } }
I20260811 09:19:08.228778 13498 leader_election.cc:288] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061), b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405)
W20260811 09:19:08.229828 12780 leader_election.cc:341] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b92630152cec469ebc7118282dfbeee2 (127.17.16.66:34405): Not found: Tablet not found: a1cd60ac63044d818106e61a4ff3d33c
W20260811 09:19:08.230039 12780 leader_election.cc:341] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b358b25eee3541609d8800dd14f09e48 (127.17.16.67:34061): Not found: Tablet not found: a1cd60ac63044d818106e61a4ff3d33c
I20260811 09:19:08.230166 12780 leader_election.cc:302] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2dec3e1860984249950375bf6012fbdd; no voters: b358b25eee3541609d8800dd14f09e48, b92630152cec469ebc7118282dfbeee2
I20260811 09:19:08.230422 13498 raft_consensus.cc:2726] T a1cd60ac63044d818106e61a4ff3d33c P 2dec3e1860984249950375bf6012fbdd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:19:08.335523 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 12680
I20260811 09:19:08.350009 13500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:08.336655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.17.16.65:42121)
0811 09:19:08.337365 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:08.337375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:08.337410 (+ 35us) server_negotiation.cc:407] Connection header received
0811 09:19:08.337489 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:08.337498 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:08.337806 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:08.337984 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:08.339316 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:08.340360 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:08.341552 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:08.342102 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:08.342210 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:08.344625 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:08.344648 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:08.344663 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:08.344701 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:08.347555 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:08.348174 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:08.348180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:08.348183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:08.348321 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:08.348588 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:08.348591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:08.348593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:08.348993 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:08.349238 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:08.349444 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 09:19:08.349655 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":348,"threads_started":1}
I20260811 09:19:08.351296 12955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:08.352691 13508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:08.351809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:35519 (local address 127.17.16.67:43047)
0811 09:19:08.352348 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:08.352501 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.126:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":250,"threads_started":1}
I20260811 09:19:08.438851 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 12680
I20260811 09:19:08.465031 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 12975
I20260811 09:19:08.477718 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:08.465794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.17.16.66:34405)
0811 09:19:08.466330 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:08.466334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:08.466351 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:19:08.466408 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:08.466412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:08.466470 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:08.466559 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:08.467871 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:08.468750 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:08.470078 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:08.470504 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:08.470599 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:08.473147 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:08.473152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:08.473155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:08.473187 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:08.475509 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:08.476060 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:08.476064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:08.476066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:08.476204 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:08.476484 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:08.476487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:08.476489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:08.476742 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:08.476974 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:08.477181 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 09:19:08.477345 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":320,"threads_started":1}
I20260811 09:19:08.478374 13174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:08.559918 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 12975
I20260811 09:19:08.580397 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 13184
I20260811 09:19:08.593042 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:08.581210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.17.16.67:34061)
0811 09:19:08.581741 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:08.581745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:08.581762 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:19:08.581820 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:08.581823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:08.581892 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:08.581981 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:08.583385 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:08.584176 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:08.585431 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:08.585854 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:08.585944 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:08.588351 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:08.588364 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:08.588367 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:08.588400 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:08.590830 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:08.591405 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:08.591408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:08.591410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:08.591551 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:08.591823 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:08.591825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:08.591827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:08.592063 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:08.592266 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:08.592450 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 09:19:08.592615 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":247,"threads_started":1}
I20260811 09:19:08.593739 13397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:08.675961 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 13184
2026-08-11T09:19:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64749 ms)
[----------] 30 tests from SecurityITest (299395 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:08 dist-test-slave-ncp9 krb5kdc[13517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:08 dist-test-slave-ncp9 krb5kdc[13517](info): set up 2 sockets
Aug 11 09:19:08 dist-test-slave-ncp9 krb5kdc[13517](info): commencing operation
krb5kdc: starting...
W20260811 09:19:09.945060 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:10 dist-test-slave-ncp9 krb5kdc[13517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:19:10.254498 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:44513
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:44513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:10.589427 13532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:10.589702 13532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:10.589830 13532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:10.596828 13532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:10.596959 13532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:10.597043 13532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:10.597118 13532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:10.597175 13532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:10.606755 13532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:44513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:44513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:10.607992 13532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:10.609357 13532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:10.621320 13540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:10.621359 13543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:10.621394 13541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:10.621754 13532 server_base.cc:1034] running on GCE node
I20260811 09:19:10.622793 13532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:10.624219 13532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:10.625448 13532 hybrid_clock.cc:648] HybridClock initialized: now 1786439950625391 us; error 41 us; skew 500 ppm
I20260811 09:19:10.625808 13532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:10 dist-test-slave-ncp9 krb5kdc[13517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:10.628818 13532 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:19:10.630275 13532 webserver.cc:460] Webserver started at http://127.17.16.126:45039/ using document root <none> and password file <none>
I20260811 09:19:10.630960 13532 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:10.631086 13532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:10.631382 13532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:10.634492 13532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "0224756ec3a24a3c88a56f4fe7dbedba"
format_stamp: "Formatted at 2026-08-11 09:19:10 on dist-test-slave-ncp9"
server_key: "d2ddbc7c4bab61adcbf8ea49a780541d"
server_key_iv: "541cd98a5ef3c4b77327dc82c98feeea"
server_key_version: "encryptionkey@0"
I20260811 09:19:10.635236 13532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "0224756ec3a24a3c88a56f4fe7dbedba"
format_stamp: "Formatted at 2026-08-11 09:19:10 on dist-test-slave-ncp9"
server_key: "d2ddbc7c4bab61adcbf8ea49a780541d"
server_key_iv: "541cd98a5ef3c4b77327dc82c98feeea"
server_key_version: "encryptionkey@0"
I20260811 09:19:10.640280 13532 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 09:19:10.643711 13556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:10.645025 13532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:19:10.645190 13532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "0224756ec3a24a3c88a56f4fe7dbedba"
format_stamp: "Formatted at 2026-08-11 09:19:10 on dist-test-slave-ncp9"
server_key: "d2ddbc7c4bab61adcbf8ea49a780541d"
server_key_iv: "541cd98a5ef3c4b77327dc82c98feeea"
server_key_version: "encryptionkey@0"
I20260811 09:19:10.645370 13532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:10.665946 13532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:10.669807 13532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:10.670068 13532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:10.687954 13532 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:44513
I20260811 09:19:10.687965 13633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:44513 every 8 connection(s)
I20260811 09:19:10.689363 13532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:10.692981 13636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:10.693686 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 13532
I20260811 09:19:10.693876 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:10.694548 17473 external_mini_cluster.cc:1428] Setting key f8f7965661814b87e1d2c0638daa7e37
I20260811 09:19:10.707053 13636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba: Bootstrap starting.
Aug 11 09:19:10 dist-test-slave-ncp9 krb5kdc[13517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:10.711028 13636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:10.712101 13636 log.cc:824] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:10.715267 13636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba: No bootstrap required, opened a new log
I20260811 09:19:10.718578 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:10.696701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.17.16.126:44513)
0811 09:19:10.697494 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:10.697506 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:10.697547 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:19:10.699273 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:10.699300 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:10.699690 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:10.700021 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:10.701869 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:10.703189 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:10.705160 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:10.705814 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:10.705939 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:10.711152 (+ 5213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:10.711164 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:10.711180 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:10.711218 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:10.714374 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:10.715182 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:10.715187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:10.715194 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:10.715382 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:10.715788 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:10.715792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:10.715794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:10.716316 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:10.716586 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:10.717203 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 09:19:10.717600 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":374,"threads_started":1}
I20260811 09:19:10.724042 13636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0224756ec3a24a3c88a56f4fe7dbedba" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44513 } }
I20260811 09:19:10.724417 13636 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:10.724516 13636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0224756ec3a24a3c88a56f4fe7dbedba, State: Initialized, Role: FOLLOWER
I20260811 09:19:10.725165 13636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0224756ec3a24a3c88a56f4fe7dbedba" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44513 } }
I20260811 09:19:10.725405 13636 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:10.725538 13636 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:10.725689 13636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:10.727850 13636 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0224756ec3a24a3c88a56f4fe7dbedba" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44513 } }
I20260811 09:19:10.728344 13636 leader_election.cc:302] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0224756ec3a24a3c88a56f4fe7dbedba; no voters:
I20260811 09:19:10.728946 13636 leader_election.cc:288] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:10.729074 13645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:10.729718 13645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [term 1 LEADER]: Becoming Leader. State: Replica: 0224756ec3a24a3c88a56f4fe7dbedba, State: Running, Role: LEADER
I20260811 09:19:10.730329 13645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0224756ec3a24a3c88a56f4fe7dbedba" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44513 } }
I20260811 09:19:10.731139 13636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:10.733222 13648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0224756ec3a24a3c88a56f4fe7dbedba. Latest consensus state: current_term: 1 leader_uuid: "0224756ec3a24a3c88a56f4fe7dbedba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0224756ec3a24a3c88a56f4fe7dbedba" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44513 } } }
I20260811 09:19:10.733184 13646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0224756ec3a24a3c88a56f4fe7dbedba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0224756ec3a24a3c88a56f4fe7dbedba" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44513 } } }
I20260811 09:19:10.733631 13648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:10.733628 13646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:10.735330 13655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:10.740319 13655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:10.751375 13655 catalog_manager.cc:1347] Generated new cluster ID: 8135f986618e4a568cffb3e40326c368
I20260811 09:19:10.751489 13655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:10.766844 13655 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:10.768157 13655 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:10.775427 13655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba: Generated new TSK 0
I20260811 09:19:10.776266 13655 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:19:10.868153 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44513
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:11.262300 13683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:11.262575 13683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:11.262688 13683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:11.269876 13683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:11.269989 13683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:11.270114 13683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:19:11.279853 13683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:11.281108 13683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:11.282475 13683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:11.294863 13696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:11.294906 13693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:11.294927 13692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:11.295717 13683 server_base.cc:1034] running on GCE node
I20260811 09:19:11.296305 13683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:11.297228 13683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:11.298427 13683 hybrid_clock.cc:648] HybridClock initialized: now 1786439951298390 us; error 47 us; skew 500 ppm
I20260811 09:19:11.298734 13683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:11 dist-test-slave-ncp9 krb5kdc[13517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439951, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:11.301739 13683 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:19:11.303165 13683 webserver.cc:460] Webserver started at http://127.17.16.65:46587/ using document root <none> and password file <none>
I20260811 09:19:11.303794 13683 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:11.303931 13683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:11.304214 13683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:11.307060 13683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "63bc510b32a2411d9c44d6880b6d1fed"
format_stamp: "Formatted at 2026-08-11 09:19:11 on dist-test-slave-ncp9"
server_key: "59ef686024e941ed979f3532b290ea03"
server_key_iv: "05a9e0eae03662c0258371989389f243"
server_key_version: "encryptionkey@0"
I20260811 09:19:11.307783 13683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "63bc510b32a2411d9c44d6880b6d1fed"
format_stamp: "Formatted at 2026-08-11 09:19:11 on dist-test-slave-ncp9"
server_key: "59ef686024e941ed979f3532b290ea03"
server_key_iv: "05a9e0eae03662c0258371989389f243"
server_key_version: "encryptionkey@0"
I20260811 09:19:11.312704 13683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:19:11.316006 13707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:11.317410 13683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:19:11.317566 13683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "63bc510b32a2411d9c44d6880b6d1fed"
format_stamp: "Formatted at 2026-08-11 09:19:11 on dist-test-slave-ncp9"
server_key: "59ef686024e941ed979f3532b290ea03"
server_key_iv: "05a9e0eae03662c0258371989389f243"
server_key_version: "encryptionkey@0"
I20260811 09:19:11.317781 13683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:11.333930 13683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:11.337718 13683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:11.337977 13683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:11.338879 13683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:11.340152 13683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:11.340229 13683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:11.340345 13683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:11.340399 13683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:11.370368 13683 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:33861
I20260811 09:19:11.370419 13882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:33861 every 8 connection(s)
I20260811 09:19:11.371591 13683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:19:11.375411 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 13683
I20260811 09:19:11.375633 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:19:11.376266 17473 external_mini_cluster.cc:1428] Setting key 73c5424a0ec36bc7bdb51f1898bac029
Aug 11 09:19:11 dist-test-slave-ncp9 krb5kdc[13517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439951, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:11.391113 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:11.373955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:56017 (local address 127.17.16.126:44513)
0811 09:19:11.374387 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:11.374392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:11.375304 (+ 912us) server_negotiation.cc:407] Connection header received
0811 09:19:11.377283 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:11.377288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:11.377377 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:11.377472 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:11.379070 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:11.379864 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:11.381413 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:11.381883 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:11.382000 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:11.385592 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:11.385598 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:11.385601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:11.385637 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:11.388024 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:11.388816 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:11.388821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:11.388825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:11.388978 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:11.389418 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:11.389421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:11.389424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:11.389724 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:11.389853 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:11.390609 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 09:19:11.390809 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":221,"threads_started":1}
I20260811 09:19:11.392184 13887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:11.374490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:44513 (local address 127.17.16.65:56017)
0811 09:19:11.375154 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:11.375191 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:11.377006 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:11.377721 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:11.377729 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:11.378355 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:11.378904 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:11.378923 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:11.380025 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:11.380029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:11.381279 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:11.381292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:11.382108 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:11.382116 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:11.382378 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:11.383027 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:19:11.383050 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:11.385373 (+ 2323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:11.388163 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:11.388172 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:11.388188 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:11.388712 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:11.389085 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:11.389090 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:11.389093 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:11.389326 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:11.389838 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:11.389845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:11.390245 (+ 400us) client_negotiation.cc:769] Sending connection context
0811 09:19:11.390514 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 09:19:11.391004 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":251,"threads_started":1}
I20260811 09:19:11.393668 13884 heartbeater.cc:347] Connected to a master server at 127.17.16.126:44513
I20260811 09:19:11.393955 13884 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:11.394663 13884 heartbeater.cc:511] Master 127.17.16.126:44513 requested a full tablet report, sending...
I20260811 09:19:11.396651 13580 ts_manager.cc:194] Registered new tserver with Master: 63bc510b32a2411d9c44d6880b6d1fed (127.17.16.65:33861)
I20260811 09:19:11.398102 13580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:56017
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:19:11.444316 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44513
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:11.776979 13894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:11.777259 13894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:11.777396 13894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:11.784798 13894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:11.784917 13894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:11.785106 13894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:19:11.794492 13894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:11.795733 13894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:11.797075 13894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:11.809656 13902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:11.809648 13903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:11.809991 13906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:11.811100 13894 server_base.cc:1034] running on GCE node
I20260811 09:19:11.811758 13894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:11.812757 13894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:11.814002 13894 hybrid_clock.cc:648] HybridClock initialized: now 1786439951813924 us; error 52 us; skew 500 ppm
I20260811 09:19:11.814337 13894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:11 dist-test-slave-ncp9 krb5kdc[13517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439951, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:11.817615 13894 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:19:11.819043 13894 webserver.cc:460] Webserver started at http://127.17.16.66:45767/ using document root <none> and password file <none>
I20260811 09:19:11.819733 13894 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:11.819854 13894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:11.820165 13894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:11.823172 13894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "220e863ec61647ea893c820736f83e0e"
format_stamp: "Formatted at 2026-08-11 09:19:11 on dist-test-slave-ncp9"
server_key: "c7a629e3285174462709f0ab040302ea"
server_key_iv: "a67ea5b734875ac3171281be4d7abc50"
server_key_version: "encryptionkey@0"
I20260811 09:19:11.823907 13894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "220e863ec61647ea893c820736f83e0e"
format_stamp: "Formatted at 2026-08-11 09:19:11 on dist-test-slave-ncp9"
server_key: "c7a629e3285174462709f0ab040302ea"
server_key_iv: "a67ea5b734875ac3171281be4d7abc50"
server_key_version: "encryptionkey@0"
I20260811 09:19:11.828995 13894 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 09:19:11.832355 13926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:11.833722 13894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:19:11.833874 13894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "220e863ec61647ea893c820736f83e0e"
format_stamp: "Formatted at 2026-08-11 09:19:11 on dist-test-slave-ncp9"
server_key: "c7a629e3285174462709f0ab040302ea"
server_key_iv: "a67ea5b734875ac3171281be4d7abc50"
server_key_version: "encryptionkey@0"
I20260811 09:19:11.834053 13894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:11.849407 13894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:11.853134 13894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:11.853394 13894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:11.854305 13894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:11.855588 13894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:11.855676 13894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:11.855793 13894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:11.855851 13894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:11.885617 13894 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:44595
I20260811 09:19:11.885659 14123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:44595 every 8 connection(s)
I20260811 09:19:11.887269 13894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:19:11.894012 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 13894
I20260811 09:19:11.894207 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:19:11.894616 17473 external_mini_cluster.cc:1428] Setting key ed8c03c9027b5e6c0d23da812e2928c0
Aug 11 09:19:11 dist-test-slave-ncp9 krb5kdc[13517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439951, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:11.906559 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:11.890022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:38069 (local address 127.17.16.126:44513)
0811 09:19:11.890213 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:11.890219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:11.891416 (+ 1197us) server_negotiation.cc:407] Connection header received
0811 09:19:11.893192 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:11.893197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:11.893267 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:11.893351 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:11.894832 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:11.895714 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:11.897388 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:11.897826 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:11.897938 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:11.901293 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:11.901298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:11.901301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:11.901333 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:11.903657 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:11.904348 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:11.904352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:11.904354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:11.904479 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:11.904918 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:11.904923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:11.904926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:11.905201 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:11.905349 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:11.906008 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 09:19:11.906234 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:19:11.907516 14131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:11.890519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:44513 (local address 127.17.16.66:38069)
0811 09:19:11.891256 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:11.891301 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:11.892934 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:11.893531 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:11.893539 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:11.894041 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:11.894657 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:11.894674 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:11.895870 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:11.895874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:11.897251 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:11.897265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:11.898072 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:11.898082 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:11.898305 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:11.898942 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:19:11.898964 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:11.901075 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:11.903793 (+ 2718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:11.903802 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:11.903817 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:11.904227 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:11.904583 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:11.904589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:11.904592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:11.904775 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:11.905328 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:11.905336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:11.905650 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 09:19:11.905875 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 09:19:11.906394 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":279,"threads_started":1}
I20260811 09:19:11.908878 14124 heartbeater.cc:347] Connected to a master server at 127.17.16.126:44513
I20260811 09:19:11.909195 14124 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:11.909866 14124 heartbeater.cc:511] Master 127.17.16.126:44513 requested a full tablet report, sending...
I20260811 09:19:11.911329 13580 ts_manager.cc:194] Registered new tserver with Master: 220e863ec61647ea893c820736f83e0e (127.17.16.66:44595)
I20260811 09:19:11.912106 13580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:38069
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:19:11.959544 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44513
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:12.289542 14135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:12.289798 14135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:12.289930 14135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:12.297175 14135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:12.297283 14135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:12.297436 14135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:19:12.307016 14135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:44513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:12.308261 14135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:12.309576 14135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:12.322264 14145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:12.322341 14144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:12.322365 14151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:12.328510 14135 server_base.cc:1034] running on GCE node
I20260811 09:19:12.329209 14135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:12.330175 14135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:12.331380 14135 hybrid_clock.cc:648] HybridClock initialized: now 1786439952331333 us; error 41 us; skew 500 ppm
I20260811 09:19:12.331715 14135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:12 dist-test-slave-ncp9 krb5kdc[13517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439952, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:12.334806 14135 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:19:12.336304 14135 webserver.cc:460] Webserver started at http://127.17.16.67:40063/ using document root <none> and password file <none>
I20260811 09:19:12.337067 14135 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:12.337191 14135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:12.337499 14135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:12.340435 14135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "badba06d124f44e78413322ef2c9ee85"
format_stamp: "Formatted at 2026-08-11 09:19:12 on dist-test-slave-ncp9"
server_key: "aee40cd89b68634af02eb081907f5c5e"
server_key_iv: "a37da51da097b328709b87a31fc5f7e1"
server_key_version: "encryptionkey@0"
I20260811 09:19:12.341216 14135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "badba06d124f44e78413322ef2c9ee85"
format_stamp: "Formatted at 2026-08-11 09:19:12 on dist-test-slave-ncp9"
server_key: "aee40cd89b68634af02eb081907f5c5e"
server_key_iv: "a37da51da097b328709b87a31fc5f7e1"
server_key_version: "encryptionkey@0"
I20260811 09:19:12.346236 14135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:19:12.349638 14163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:12.350925 14135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:19:12.351056 14135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "badba06d124f44e78413322ef2c9ee85"
format_stamp: "Formatted at 2026-08-11 09:19:12 on dist-test-slave-ncp9"
server_key: "aee40cd89b68634af02eb081907f5c5e"
server_key_iv: "a37da51da097b328709b87a31fc5f7e1"
server_key_version: "encryptionkey@0"
I20260811 09:19:12.351229 14135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:12.363332 14135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:12.366995 14135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:12.367256 14135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:12.368115 14135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:12.369418 14135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:12.369491 14135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:12.369583 14135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:12.369625 14135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:12.399026 14135 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:43745
I20260811 09:19:12.399071 14355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:43745 every 8 connection(s)
I20260811 09:19:12.400252 14135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:19:12.400789 13884 heartbeater.cc:503] Master 127.17.16.126:44513 was elected leader, sending a full tablet report...
I20260811 09:19:12.407109 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 14135
I20260811 09:19:12.407264 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:19:12.407703 17473 external_mini_cluster.cc:1428] Setting key 84ce26f2b1424960da049aabba557674
Aug 11 09:19:12 dist-test-slave-ncp9 krb5kdc[13517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439952, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:12.419776 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.403084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:54281 (local address 127.17.16.126:44513)
0811 09:19:12.403252 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:12.403257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:12.404378 (+ 1121us) server_negotiation.cc:407] Connection header received
0811 09:19:12.406196 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:12.406201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:12.406298 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:12.406386 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:12.407914 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.408777 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.410883 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.411293 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.411386 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.414598 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:12.414603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:12.414606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:12.414638 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:12.416920 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:12.417590 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:12.417595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:12.417597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:12.417724 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:12.418128 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:12.418132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:12.418134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:12.418398 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:12.418534 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:12.419219 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 09:19:12.419441 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:19:12.420672 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.403567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:44513 (local address 127.17.16.67:54281)
0811 09:19:12.404238 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:12.404274 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:12.405940 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:12.406619 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:12.406627 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:12.407128 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:12.407728 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.407750 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.408994 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.409000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.410758 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.410776 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.411455 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.411459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.411658 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.412271 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:19:12.412294 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:12.414408 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:12.417041 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:12.417050 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:12.417066 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:12.417469 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:12.417828 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:12.417833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:12.417836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:12.418014 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:12.418520 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:12.418528 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:12.418841 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 09:19:12.419068 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 09:19:12.419566 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":249,"threads_started":1}
I20260811 09:19:12.422112 14358 heartbeater.cc:347] Connected to a master server at 127.17.16.126:44513
I20260811 09:19:12.422449 14358 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:12.423275 14358 heartbeater.cc:511] Master 127.17.16.126:44513 requested a full tablet report, sending...
I20260811 09:19:12.424606 13578 ts_manager.cc:194] Registered new tserver with Master: badba06d124f44e78413322ef2c9ee85 (127.17.16.67:43745)
I20260811 09:19:12.425570 13578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:54281
I20260811 09:19:12.434999 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:19:12.451120 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.439348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.17.16.126:44513)
0811 09:19:12.439532 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:12.439539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:12.439926 (+ 387us) server_negotiation.cc:407] Connection header received
0811 09:19:12.440046 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:12.440050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:12.440118 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:12.440203 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:12.441647 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.442461 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.443707 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.444093 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.444184 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.446433 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:12.446438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:12.446441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:12.446468 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:12.448654 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:12.449316 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:12.449320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:12.449322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:12.449449 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:12.449787 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:12.449791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:12.449795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:12.450145 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:12.450342 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:12.450695 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 09:19:12.450851 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:19:12.455127 13578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:19:12.461804 13578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:19:12.498142 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.473585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36044 (local address 127.17.16.67:43745)
0811 09:19:12.489122 (+ 15537us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:12.489129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:12.489158 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:19:12.489244 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:12.489251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:12.489539 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:12.489751 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:12.490711 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.491798 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.494781 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.496764 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.496865 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.497139 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:12.497600 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 09:19:12.497790 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15379,"thread_start_us":15199,"threads_started":1}
I20260811 09:19:12.498440 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.474142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60784 (local address 127.17.16.65:33861)
0811 09:19:12.489433 (+ 15291us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:12.489443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:12.489474 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:19:12.489587 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:12.489595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:12.489827 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:12.490996 (+ 1169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:12.491263 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.492987 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.495692 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.497123 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.497254 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.497477 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:12.497956 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 09:19:12.498125 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15099,"thread_start_us":15836,"threads_started":1}
I20260811 09:19:12.498200 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.473600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:33861 (local address 127.0.0.1:60784)
0811 09:19:12.488214 (+ 14614us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:12.488234 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:12.488355 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:12.490262 (+ 1907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:12.490272 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:12.490304 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:12.490771 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.490782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.493151 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.493158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.495448 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.495464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.497326 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.497332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.497508 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.497552 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 09:19:12.497645 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 09:19:12.497781 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14468,"mutex_wait_us":2,"thread_start_us":13379,"threads_started":1}
I20260811 09:19:12.498828 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.473269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:43745 (local address 127.0.0.1:36044)
0811 09:19:12.486472 (+ 13203us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:12.486542 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:12.486746 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:12.490081 (+ 3335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:12.490089 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:12.490121 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:12.490511 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.490529 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.491973 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.491978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.494573 (+ 2595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.494592 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.497125 (+ 2533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.497132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.497363 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.497418 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 09:19:12.497551 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 09:19:12.497756 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12873,"mutex_wait_us":2,"thread_start_us":15156,"threads_started":1}
I20260811 09:19:12.502718 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.492253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:44595 (local address 127.0.0.1:36470)
0811 09:19:12.495894 (+ 3641us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:12.495912 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:12.496029 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:12.496733 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:12.496738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:12.496764 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:12.497226 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.497240 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.499059 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.499064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.500676 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.500689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.501991 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.501997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.502161 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.502186 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:19:12.502279 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 09:19:12.502400 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3519,"thread_start_us":3345,"threads_started":1}
I20260811 09:19:12.503247 14390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.488179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36470 (local address 127.17.16.66:44595)
0811 09:19:12.495556 (+ 7377us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:12.495565 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:12.496165 (+ 600us) server_negotiation.cc:407] Connection header received
0811 09:19:12.496267 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:12.496275 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:12.496513 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:12.496719 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:12.497422 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.498915 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.500831 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.501794 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.501917 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.502109 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:12.502561 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 09:19:12.502733 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7203,"thread_start_us":7063,"threads_started":1}
I20260811 09:19:12.502970 14253 tablet_service.cc:1467] Processing CreateTablet for tablet 59e79455a29b40f4af964d4081c26a1a (DEFAULT_TABLE table=test-table [id=480609a2adc448c4aaecf417a902391f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:12.502970 13771 tablet_service.cc:1467] Processing CreateTablet for tablet 59e79455a29b40f4af964d4081c26a1a (DEFAULT_TABLE table=test-table [id=480609a2adc448c4aaecf417a902391f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:12.504575 13771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59e79455a29b40f4af964d4081c26a1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:12.504580 14253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59e79455a29b40f4af964d4081c26a1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:12.506582 14005 tablet_service.cc:1467] Processing CreateTablet for tablet 59e79455a29b40f4af964d4081c26a1a (DEFAULT_TABLE table=test-table [id=480609a2adc448c4aaecf417a902391f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:12.507877 14005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59e79455a29b40f4af964d4081c26a1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:12.518764 14395 tablet_bootstrap.cc:492] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: Bootstrap starting.
I20260811 09:19:12.519800 14396 tablet_bootstrap.cc:492] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: Bootstrap starting.
I20260811 09:19:12.521723 14397 tablet_bootstrap.cc:492] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: Bootstrap starting.
I20260811 09:19:12.523690 14396 tablet_bootstrap.cc:654] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:12.524147 14395 tablet_bootstrap.cc:654] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:12.524698 14397 tablet_bootstrap.cc:654] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:12.525046 14396 log.cc:824] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:12.525341 14395 log.cc:824] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:12.525699 14397 log.cc:824] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:12.528268 14396 tablet_bootstrap.cc:492] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: No bootstrap required, opened a new log
I20260811 09:19:12.528558 14396 ts_tablet_manager.cc:1397] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 09:19:12.528767 14395 tablet_bootstrap.cc:492] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: No bootstrap required, opened a new log
I20260811 09:19:12.529064 14395 ts_tablet_manager.cc:1397] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.006s
I20260811 09:19:12.530055 14397 tablet_bootstrap.cc:492] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: No bootstrap required, opened a new log
I20260811 09:19:12.530236 14397 ts_tablet_manager.cc:1397] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 09:19:12.537782 14397 raft_consensus.cc:348] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 } } peers { permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 } } peers { permanent_uuid: "220e863ec61647ea893c820736f83e0e" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44595 } }
I20260811 09:19:12.538172 14397 raft_consensus.cc:374] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:12.538262 14397 raft_consensus.cc:729] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: badba06d124f44e78413322ef2c9ee85, State: Initialized, Role: FOLLOWER
I20260811 09:19:12.538872 14397 consensus_queue.cc:260] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 } } peers { permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 } } peers { permanent_uuid: "220e863ec61647ea893c820736f83e0e" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44595 } }
I20260811 09:19:12.539281 14395 raft_consensus.cc:348] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 } } peers { permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 } } peers { permanent_uuid: "220e863ec61647ea893c820736f83e0e" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44595 } }
I20260811 09:19:12.539288 14396 raft_consensus.cc:348] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 } } peers { permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 } } peers { permanent_uuid: "220e863ec61647ea893c820736f83e0e" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44595 } }
I20260811 09:19:12.539710 14395 raft_consensus.cc:374] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:12.539710 14396 raft_consensus.cc:374] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:12.539814 14395 raft_consensus.cc:729] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220e863ec61647ea893c820736f83e0e, State: Initialized, Role: FOLLOWER
I20260811 09:19:12.539814 14396 raft_consensus.cc:729] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63bc510b32a2411d9c44d6880b6d1fed, State: Initialized, Role: FOLLOWER
I20260811 09:19:12.540069 14358 heartbeater.cc:503] Master 127.17.16.126:44513 was elected leader, sending a full tablet report...
I20260811 09:19:12.540403 14395 consensus_queue.cc:260] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 } } peers { permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 } } peers { permanent_uuid: "220e863ec61647ea893c820736f83e0e" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44595 } }
I20260811 09:19:12.540704 14397 ts_tablet_manager.cc:1428] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260811 09:19:12.540607 14396 consensus_queue.cc:260] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 } } peers { permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 } } peers { permanent_uuid: "220e863ec61647ea893c820736f83e0e" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44595 } }
I20260811 09:19:12.542667 14396 ts_tablet_manager.cc:1428] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 09:19:12.543624 14124 heartbeater.cc:503] Master 127.17.16.126:44513 was elected leader, sending a full tablet report...
I20260811 09:19:12.544310 14395 ts_tablet_manager.cc:1428] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
W20260811 09:19:12.623778 13885 tablet.cc:2369] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:19:12.639626 14130 tablet.cc:2369] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:19:12.642817 14402 raft_consensus.cc:482] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:19:12.643108 14402 raft_consensus.cc:504] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 } } peers { permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 } } peers { permanent_uuid: "220e863ec61647ea893c820736f83e0e" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44595 } }
I20260811 09:19:12.644815 14402 leader_election.cc:288] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers badba06d124f44e78413322ef2c9ee85 (127.17.16.67:43745), 63bc510b32a2411d9c44d6880b6d1fed (127.17.16.65:33861)
I20260811 09:19:12.651381 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.645231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:47765 (local address 127.17.16.65:33861)
0811 09:19:12.645450 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:12.645456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:12.646073 (+ 617us) server_negotiation.cc:407] Connection header received
0811 09:19:12.646288 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:12.646292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:12.646354 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:12.646428 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:12.646961 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.647690 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.649649 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.650489 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.650650 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.650728 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:12.651016 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 09:19:12.651143 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:19:12.651553 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.645231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:50373 (local address 127.17.16.67:43745)
0811 09:19:12.645450 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:12.645456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:12.646455 (+ 999us) server_negotiation.cc:407] Connection header received
0811 09:19:12.646624 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:12.646628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:12.646689 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:12.646759 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:12.647301 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.648034 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.649840 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.650632 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.650777 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.650847 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:12.651168 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 09:19:12.651290 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:19:12.651569 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.645284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:33861 (local address 127.17.16.66:47765)
0811 09:19:12.645955 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:12.645970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:12.646124 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:12.646463 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:12.646467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:12.646494 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:12.646816 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.646826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.647818 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.647822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.649484 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.649496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.650701 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.650707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.650843 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.650890 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 09:19:12.650975 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 09:19:12.651107 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":26,"thread_start_us":407,"threads_started":1}
W20260811 09:19:12.652038 14361 tablet.cc:2369] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:19:12.652277 14407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.645325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:43745 (local address 127.17.16.66:50373)
0811 09:19:12.646352 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:12.646367 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:12.646491 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:12.646804 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:12.646809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:12.646833 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:12.647158 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.647167 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.648166 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.648170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.649701 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:12.649713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.650835 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.650841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:12.651027 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.651052 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:19:12.651129 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 09:19:12.651250 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":44,"spinlock_wait_cycles":22912,"thread_start_us":722,"threads_started":1}
I20260811 09:19:12.652563 13812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59e79455a29b40f4af964d4081c26a1a" candidate_uuid: "220e863ec61647ea893c820736f83e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63bc510b32a2411d9c44d6880b6d1fed" is_pre_election: true
I20260811 09:19:12.653164 13812 raft_consensus.cc:2454] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 220e863ec61647ea893c820736f83e0e in term 0.
I20260811 09:19:12.653265 14288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59e79455a29b40f4af964d4081c26a1a" candidate_uuid: "220e863ec61647ea893c820736f83e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "badba06d124f44e78413322ef2c9ee85" is_pre_election: true
I20260811 09:19:12.653743 14288 raft_consensus.cc:2454] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 220e863ec61647ea893c820736f83e0e in term 0.
I20260811 09:19:12.653869 13931 leader_election.cc:302] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220e863ec61647ea893c820736f83e0e, 63bc510b32a2411d9c44d6880b6d1fed; no voters:
I20260811 09:19:12.654263 14402 raft_consensus.cc:2781] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:19:12.654400 14402 raft_consensus.cc:482] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:19:12.654495 14402 raft_consensus.cc:3037] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:12.656693 14402 raft_consensus.cc:504] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 } } peers { permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 } } peers { permanent_uuid: "220e863ec61647ea893c820736f83e0e" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44595 } }
I20260811 09:19:12.657266 14402 leader_election.cc:288] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [CANDIDATE]: Term 1 election: Requested vote from peers badba06d124f44e78413322ef2c9ee85 (127.17.16.67:43745), 63bc510b32a2411d9c44d6880b6d1fed (127.17.16.65:33861)
I20260811 09:19:12.657701 14288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59e79455a29b40f4af964d4081c26a1a" candidate_uuid: "220e863ec61647ea893c820736f83e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "badba06d124f44e78413322ef2c9ee85"
I20260811 09:19:12.657737 13812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59e79455a29b40f4af964d4081c26a1a" candidate_uuid: "220e863ec61647ea893c820736f83e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63bc510b32a2411d9c44d6880b6d1fed"
I20260811 09:19:12.657917 14288 raft_consensus.cc:3037] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:12.657923 13812 raft_consensus.cc:3037] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:12.661084 13812 raft_consensus.cc:2454] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 220e863ec61647ea893c820736f83e0e in term 1.
I20260811 09:19:12.661084 14288 raft_consensus.cc:2454] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 220e863ec61647ea893c820736f83e0e in term 1.
I20260811 09:19:12.661456 13928 leader_election.cc:302] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220e863ec61647ea893c820736f83e0e, badba06d124f44e78413322ef2c9ee85; no voters:
I20260811 09:19:12.661737 14402 raft_consensus.cc:2781] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:12.662223 14402 raft_consensus.cc:686] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 1 LEADER]: Becoming Leader. State: Replica: 220e863ec61647ea893c820736f83e0e, State: Running, Role: LEADER
I20260811 09:19:12.662768 14402 consensus_queue.cc:237] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 } } peers { permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 } } peers { permanent_uuid: "220e863ec61647ea893c820736f83e0e" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44595 } }
I20260811 09:19:12.667445 13580 catalog_manager.cc:5495] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e reported cstate change: term changed from 0 to 1, leader changed from <none> to 220e863ec61647ea893c820736f83e0e (127.17.16.66). New cstate: current_term: 1 leader_uuid: "220e863ec61647ea893c820736f83e0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "220e863ec61647ea893c820736f83e0e" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 44595 } health_report { overall_health: HEALTHY } } }
I20260811 09:19:12.721690 14390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.716048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36480 (local address 127.17.16.66:44595)
0811 09:19:12.716246 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:12.716252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:12.716317 (+ 65us) server_negotiation.cc:407] Connection header received
0811 09:19:12.716576 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:12.716580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:12.716671 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:12.716800 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:19:12.717259 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.718000 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.719434 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.719852 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.719963 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.720477 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:19:12.721096 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:19:12.721241 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:12.721333 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 09:19:12.721420 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:19:12.730146 13812 raft_consensus.cc:1261] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed [term 1 FOLLOWER]: Refusing update from remote peer 220e863ec61647ea893c820736f83e0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:12.730146 14288 raft_consensus.cc:1261] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85 [term 1 FOLLOWER]: Refusing update from remote peer 220e863ec61647ea893c820736f83e0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:12.731348 14402 consensus_queue.cc:1035] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "badba06d124f44e78413322ef2c9ee85" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:12.731784 14408 consensus_queue.cc:1035] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "63bc510b32a2411d9c44d6880b6d1fed" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:12.745550 14416 mvcc.cc:205] Tried to move back new op lower bound from 7317258046371966976 to 7317258046110855168. Current Snapshot: MvccSnapshot[applied={T|T < 7317258046371966976}]
I20260811 09:19:12.759302 13578 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:12.759603 13578 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:12.763159 13578 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0224756ec3a24a3c88a56f4fe7dbedba: Sending DeleteTablet for 3 replicas of tablet 59e79455a29b40f4af964d4081c26a1a
I20260811 09:19:12.764325 14253 tablet_service.cc:1514] Processing DeleteTablet for tablet 59e79455a29b40f4af964d4081c26a1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:12 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:36044
I20260811 09:19:12.764645 13771 tablet_service.cc:1514] Processing DeleteTablet for tablet 59e79455a29b40f4af964d4081c26a1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:12 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:60784
I20260811 09:19:12.764834 14005 tablet_service.cc:1514] Processing DeleteTablet for tablet 59e79455a29b40f4af964d4081c26a1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:12 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:36470
I20260811 09:19:12.766623 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 13683
I20260811 09:19:12.778806 14433 tablet_replica.cc:330] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: stopping tablet replica
I20260811 09:19:12.779572 14431 tablet_replica.cc:330] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: stopping tablet replica
I20260811 09:19:12.780046 14431 raft_consensus.cc:2229] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:12.780606 14431 raft_consensus.cc:2258] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:12.782577 14433 raft_consensus.cc:2229] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:19:12.783109 14431 ts_tablet_manager.cc:1905] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:12.783752 14432 tablet_replica.cc:330] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: stopping tablet replica
I20260811 09:19:12.784241 14432 raft_consensus.cc:2229] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:12.784111 14433 raft_consensus.cc:2258] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:12.784863 14432 raft_consensus.cc:2258] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:12.787083 14432 ts_tablet_manager.cc:1905] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:19:12 dist-test-slave-ncp9 krb5kdc[13517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:19:12.790179 14433 ts_tablet_manager.cc:1905] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:12.794226 14432 ts_tablet_manager.cc:1918] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:12.794485 14432 log.cc:1196] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/59e79455a29b40f4af964d4081c26a1a
I20260811 09:19:12.795609 14432 ts_tablet_manager.cc:1939] T 59e79455a29b40f4af964d4081c26a1a P 63bc510b32a2411d9c44d6880b6d1fed: Deleting consensus metadata
I20260811 09:19:12.797739 13562 catalog_manager.cc:4841] TS 63bc510b32a2411d9c44d6880b6d1fed (127.17.16.65:33861): tablet 59e79455a29b40f4af964d4081c26a1a (table test-table [id=480609a2adc448c4aaecf417a902391f]) successfully deleted
I20260811 09:19:12.798596 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.768144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60792 (local address 127.17.16.65:33861)
0811 09:19:12.768320 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:12.768326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:12.773551 (+ 5225us) server_negotiation.cc:407] Connection header received
0811 09:19:12.774482 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:12.774488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:12.774592 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:12.774666 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:12.779188 (+ 4522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.780354 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.783812 (+ 3458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.784527 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.784662 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.790533 (+ 5871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:12.790549 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:12.790567 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:12.790613 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:12.794740 (+ 4127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:12.795730 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:12.795740 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:12.795744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:12.795932 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:12.796427 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:12.796433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:12.796436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:12.796991 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:12.797311 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:12.797969 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 09:19:12.798207 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:19:12.798750 14431 ts_tablet_manager.cc:1918] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:12.798918 14431 log.cc:1196] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/59e79455a29b40f4af964d4081c26a1a
I20260811 09:19:12.799100 14433 ts_tablet_manager.cc:1918] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:12.799316 14433 log.cc:1196] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/59e79455a29b40f4af964d4081c26a1a
I20260811 09:19:12.799495 14431 ts_tablet_manager.cc:1939] T 59e79455a29b40f4af964d4081c26a1a P badba06d124f44e78413322ef2c9ee85: Deleting consensus metadata
I20260811 09:19:12.799988 14433 ts_tablet_manager.cc:1939] T 59e79455a29b40f4af964d4081c26a1a P 220e863ec61647ea893c820736f83e0e: Deleting consensus metadata
I20260811 09:19:12.800101 13875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:12.801412 13557 catalog_manager.cc:4841] TS badba06d124f44e78413322ef2c9ee85 (127.17.16.67:43745): tablet 59e79455a29b40f4af964d4081c26a1a (table test-table [id=480609a2adc448c4aaecf417a902391f]) successfully deleted
I20260811 09:19:12.801750 13562 catalog_manager.cc:4841] TS 220e863ec61647ea893c820736f83e0e (127.17.16.66:44595): tablet 59e79455a29b40f4af964d4081c26a1a (table test-table [id=480609a2adc448c4aaecf417a902391f]) successfully deleted
I20260811 09:19:12.891038 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 13683
I20260811 09:19:12.918493 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 13894
Aug 11 09:19:12 dist-test-slave-ncp9 krb5kdc[13517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:19:12.936568 14390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:12.919212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.17.16.66:44595)
0811 09:19:12.919395 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:12.919401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:12.919551 (+ 150us) server_negotiation.cc:407] Connection header received
0811 09:19:12.919733 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:12.919737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:12.919812 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:12.919899 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:12.921296 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.922605 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.923944 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:12.924600 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:12.924729 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:12.929359 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:12.929377 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:12.929403 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:12.929449 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:12.933556 (+ 4107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:12.934330 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:12.934340 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:12.934344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:12.934532 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:12.934877 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:12.934882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:12.934886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:12.935393 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:12.935758 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:12.935916 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 09:19:12.936151 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:19:12.937458 14111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:13.051342 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 13894
I20260811 09:19:13.074987 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 14135
Aug 11 09:19:13 dist-test-slave-ncp9 krb5kdc[13517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:19:13.094449 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:13.075729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36058 (local address 127.17.16.67:43745)
0811 09:19:13.075917 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:13.075923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:13.075944 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:19:13.076149 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:13.076153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:13.076220 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:13.076305 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:13.077794 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:13.078513 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:13.079816 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:13.080216 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:13.080324 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:13.089069 (+ 8745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:13.089091 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:13.089113 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:13.089166 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:13.091630 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:13.092322 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:13.092330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:13.092332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:13.092479 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:13.092834 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:13.092838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:13.092840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:13.093242 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:13.093527 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:13.093895 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 09:19:13.094132 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:19:13.095093 14349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:13.183265 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 14135
I20260811 09:19:13.204949 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 13532
I20260811 09:19:13.206101 13628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:13.316439 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 13532
2026-08-11T09:19:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4650 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T09:19:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:13Z Disabled control of system clock
I20260811 09:19:13.418439 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:43655
--webserver_interface=127.17.16.126
--webserver_port=0
--builtin_ntp_servers=127.17.16.84:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:43655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:19:13.757714 14446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:13.758004 14446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:13.758117 14446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:13.765359 14446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:13.765470 14446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:13.765508 14446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:13.765574 14446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:13.765641 14446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:13.774916 14446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:43655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:43655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:13.776211 14446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:13.778092 14446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:13.789680 14453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:13.789731 14454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:13.789769 14457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:13.790243 14446 server_base.cc:1034] running on GCE node
I20260811 09:19:13.790972 14446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:13.792299 14446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:13.793499 14446 hybrid_clock.cc:648] HybridClock initialized: now 1786439953793447 us; error 34 us; skew 500 ppm
I20260811 09:19:13.793826 14446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:13.795958 14446 webserver.cc:460] Webserver started at http://127.17.16.126:37959/ using document root <none> and password file <none>
I20260811 09:19:13.796622 14446 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:13.796722 14446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:13.797070 14446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:13.799937 14446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "fcbfb41672f24c54926d4b0d18fecfd0"
format_stamp: "Formatted at 2026-08-11 09:19:13 on dist-test-slave-ncp9"
server_key: "5dd42c66808cd2efdfaeb3a2ef3f4207"
server_key_iv: "24579d2fd81c2d35899fae2e736de7b8"
server_key_version: "encryptionkey@0"
I20260811 09:19:13.800729 14446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "fcbfb41672f24c54926d4b0d18fecfd0"
format_stamp: "Formatted at 2026-08-11 09:19:13 on dist-test-slave-ncp9"
server_key: "5dd42c66808cd2efdfaeb3a2ef3f4207"
server_key_iv: "24579d2fd81c2d35899fae2e736de7b8"
server_key_version: "encryptionkey@0"
I20260811 09:19:13.805591 14446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:19:13.809027 14469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:13.810235 14446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:19:13.810389 14446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "fcbfb41672f24c54926d4b0d18fecfd0"
format_stamp: "Formatted at 2026-08-11 09:19:13 on dist-test-slave-ncp9"
server_key: "5dd42c66808cd2efdfaeb3a2ef3f4207"
server_key_iv: "24579d2fd81c2d35899fae2e736de7b8"
server_key_version: "encryptionkey@0"
I20260811 09:19:13.810582 14446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:13.837831 14446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:13.838750 14446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:13.839043 14446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:13.856215 14446 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:43655
I20260811 09:19:13.856251 14551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:43655 every 8 connection(s)
I20260811 09:19:13.857553 14446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:13.858578 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 14446
I20260811 09:19:13.858731 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:13.859184 17473 external_mini_cluster.cc:1428] Setting key 77fe064caaa6f8c5f5849988c515682d
I20260811 09:19:13.861812 14554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:13.872613 14554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0: Bootstrap starting.
I20260811 09:19:13.873185 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:13.861049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.17.16.126:43655)
0811 09:19:13.862184 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:13.862211 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:13.862259 (+ 48us) server_negotiation.cc:407] Connection header received
0811 09:19:13.864056 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:13.864078 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:13.864576 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:13.865002 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:13.865618 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:13.867043 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:13.868521 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:13.869264 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:13.869493 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:13.870777 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:13.870786 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:13.870799 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:13.871349 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:13.871428 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:13.871464 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:13.871576 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:13.871816 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 09:19:13.872116 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":414,"threads_started":1}
I20260811 09:19:13.876237 14554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:13.877310 14554 log.cc:824] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:13.879949 14554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0: No bootstrap required, opened a new log
I20260811 09:19:13.887689 14554 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbfb41672f24c54926d4b0d18fecfd0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43655 } }
I20260811 09:19:13.888023 14554 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:13.888120 14554 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcbfb41672f24c54926d4b0d18fecfd0, State: Initialized, Role: FOLLOWER
I20260811 09:19:13.888648 14554 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbfb41672f24c54926d4b0d18fecfd0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43655 } }
I20260811 09:19:13.888855 14554 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:13.889012 14554 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:13.889169 14554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:13.891325 14554 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbfb41672f24c54926d4b0d18fecfd0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43655 } }
I20260811 09:19:13.891876 14554 leader_election.cc:302] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcbfb41672f24c54926d4b0d18fecfd0; no voters:
I20260811 09:19:13.892500 14554 leader_election.cc:288] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:13.892652 14560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:13.893484 14560 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [term 1 LEADER]: Becoming Leader. State: Replica: fcbfb41672f24c54926d4b0d18fecfd0, State: Running, Role: LEADER
I20260811 09:19:13.894028 14560 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbfb41672f24c54926d4b0d18fecfd0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43655 } }
I20260811 09:19:13.894521 14554 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:13.896853 14563 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcbfb41672f24c54926d4b0d18fecfd0. Latest consensus state: current_term: 1 leader_uuid: "fcbfb41672f24c54926d4b0d18fecfd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbfb41672f24c54926d4b0d18fecfd0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43655 } } }
I20260811 09:19:13.897279 14563 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:13.897486 14562 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcbfb41672f24c54926d4b0d18fecfd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbfb41672f24c54926d4b0d18fecfd0" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43655 } } }
I20260811 09:19:13.898082 14562 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:13.900521 14572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:13.906203 14572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:13.914937 14572 catalog_manager.cc:1347] Generated new cluster ID: 5f1d2051110e4538a13cc9e63f558692
I20260811 09:19:13.915066 14572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:13.933964 14572 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:13.935251 14572 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:13.941619 14572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0: Generated new TSK 0
I20260811 09:19:13.942416 14572 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:13.950604 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:43655
--builtin_ntp_servers=127.17.16.84:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:19:14.292302 14591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:14.292570 14591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:14.292685 14591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:14.300190 14591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:14.300299 14591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:14.300433 14591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:19:14.309940 14591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:43655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:14.311282 14591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:14.313158 14591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:14.329916 14597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:14.329914 14598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:14.330034 14604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:14.331230 14591 server_base.cc:1034] running on GCE node
I20260811 09:19:14.331877 14591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:14.333017 14591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:14.334257 14591 hybrid_clock.cc:648] HybridClock initialized: now 1786439954334185 us; error 56 us; skew 500 ppm
I20260811 09:19:14.334641 14591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:14.336881 14591 webserver.cc:460] Webserver started at http://127.17.16.65:41929/ using document root <none> and password file <none>
I20260811 09:19:14.337607 14591 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:14.337697 14591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:14.337967 14591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:14.340817 14591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "8ddfa8a2c9b34abbb238bcae67db882a"
format_stamp: "Formatted at 2026-08-11 09:19:14 on dist-test-slave-ncp9"
server_key: "83ff28f3cce3bb1ba345a0d5529478b2"
server_key_iv: "e58571c0a966297108a35c426c2a0493"
server_key_version: "encryptionkey@0"
I20260811 09:19:14.341634 14591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "8ddfa8a2c9b34abbb238bcae67db882a"
format_stamp: "Formatted at 2026-08-11 09:19:14 on dist-test-slave-ncp9"
server_key: "83ff28f3cce3bb1ba345a0d5529478b2"
server_key_iv: "e58571c0a966297108a35c426c2a0493"
server_key_version: "encryptionkey@0"
I20260811 09:19:14.346841 14591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:19:14.350322 14610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:14.351609 14591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:19:14.351766 14591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "8ddfa8a2c9b34abbb238bcae67db882a"
format_stamp: "Formatted at 2026-08-11 09:19:14 on dist-test-slave-ncp9"
server_key: "83ff28f3cce3bb1ba345a0d5529478b2"
server_key_iv: "e58571c0a966297108a35c426c2a0493"
server_key_version: "encryptionkey@0"
I20260811 09:19:14.351972 14591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:14.379815 14591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:14.380690 14591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:14.381035 14591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:14.382030 14591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:14.383623 14591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:14.383729 14591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:14.383841 14591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:14.383909 14591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:14.413753 14591 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:40053
I20260811 09:19:14.413831 14784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:40053 every 8 connection(s)
I20260811 09:19:14.415165 14591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:19:14.417524 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 14591
I20260811 09:19:14.417703 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:19:14.418279 17473 external_mini_cluster.cc:1428] Setting key a9d502d9e6c99131896f8aff78be5298
I20260811 09:19:14.420949 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:43655
--builtin_ntp_servers=127.17.16.84:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:19:14.427632 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:14.417665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:48995 (local address 127.17.16.126:43655)
0811 09:19:14.418170 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:14.418176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:14.419445 (+ 1269us) server_negotiation.cc:407] Connection header received
0811 09:19:14.421331 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:14.421337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:14.421418 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:14.421514 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:14.422622 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:14.423336 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:14.424872 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:14.425320 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:14.425431 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:14.426799 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:14.426804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:14.426807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:14.426825 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:14.426865 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:14.426877 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:14.426952 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:14.427178 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 09:19:14.427277 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":238,"threads_started":1}
I20260811 09:19:14.428671 14788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:14.418235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:43655 (local address 127.17.16.65:48995)
0811 09:19:14.419281 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:14.419335 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:14.421045 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:14.421769 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:14.421778 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:14.421864 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:14.422441 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:14.422458 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:14.423480 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:14.423495 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:14.424729 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:14.424742 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:14.425551 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:14.425558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:14.425776 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:14.426471 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:19:14.426474 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:14.426524 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:19:14.426601 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:14.426967 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:14.426979 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:14.426988 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 09:19:14.427158 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 09:19:14.427561 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":335,"threads_started":1}
I20260811 09:19:14.430147 14785 heartbeater.cc:347] Connected to a master server at 127.17.16.126:43655
I20260811 09:19:14.430426 14785 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:14.431154 14785 heartbeater.cc:511] Master 127.17.16.126:43655 requested a full tablet report, sending...
I20260811 09:19:14.433422 14491 ts_manager.cc:194] Registered new tserver with Master: 8ddfa8a2c9b34abbb238bcae67db882a (127.17.16.65:40053)
I20260811 09:19:14.434939 14491 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.65:48995
W20260811 09:19:14.756335 14789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:14.756611 14789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:14.756706 14789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:14.763736 14789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:14.763833 14789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:14.763942 14789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:19:14.772680 14789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:43655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:14.774185 14789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:14.775872 14789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:14.788833 14798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:14.788836 14796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:14.788759 14800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:14.794353 14789 server_base.cc:1034] running on GCE node
I20260811 09:19:14.794957 14789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:14.795861 14789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:14.797058 14789 hybrid_clock.cc:648] HybridClock initialized: now 1786439954797005 us; error 39 us; skew 500 ppm
I20260811 09:19:14.797365 14789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:14.799562 14789 webserver.cc:460] Webserver started at http://127.17.16.66:33975/ using document root <none> and password file <none>
I20260811 09:19:14.800257 14789 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:14.800351 14789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:14.800671 14789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:14.803596 14789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "bc0b4c1a223b420a837d8d34087100ef"
format_stamp: "Formatted at 2026-08-11 09:19:14 on dist-test-slave-ncp9"
server_key: "2bd36abcd73efaf5fcd4738a3ad8b18a"
server_key_iv: "95312efcf3513e60ebebc114f2e4bcc3"
server_key_version: "encryptionkey@0"
I20260811 09:19:14.804368 14789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "bc0b4c1a223b420a837d8d34087100ef"
format_stamp: "Formatted at 2026-08-11 09:19:14 on dist-test-slave-ncp9"
server_key: "2bd36abcd73efaf5fcd4738a3ad8b18a"
server_key_iv: "95312efcf3513e60ebebc114f2e4bcc3"
server_key_version: "encryptionkey@0"
I20260811 09:19:14.809388 14789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 09:19:14.812649 14807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:14.813934 14789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:19:14.814080 14789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "bc0b4c1a223b420a837d8d34087100ef"
format_stamp: "Formatted at 2026-08-11 09:19:14 on dist-test-slave-ncp9"
server_key: "2bd36abcd73efaf5fcd4738a3ad8b18a"
server_key_iv: "95312efcf3513e60ebebc114f2e4bcc3"
server_key_version: "encryptionkey@0"
I20260811 09:19:14.814275 14789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:14.830860 14789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:14.831658 14789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:14.831940 14789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:14.832916 14789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:14.834327 14789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:14.834425 14789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:14.834538 14789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:14.834604 14789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:14.864706 14789 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:38847
I20260811 09:19:14.864745 14990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:38847 every 8 connection(s)
I20260811 09:19:14.866281 14789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:19:14.871760 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 14789
I20260811 09:19:14.871941 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:19:14.872466 17473 external_mini_cluster.cc:1428] Setting key 01f94096fd14d0dfd6fe59a010f29ba0
I20260811 09:19:14.875416 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:43655
--builtin_ntp_servers=127.17.16.84:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:19:14.879863 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:14.868841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:52475 (local address 127.17.16.126:43655)
0811 09:19:14.869039 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:14.869045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:14.870608 (+ 1563us) server_negotiation.cc:407] Connection header received
0811 09:19:14.872437 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:14.872443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:14.872525 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:14.872622 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:14.873891 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:14.875039 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:14.877290 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:14.877682 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:14.877778 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:14.879204 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:14.879209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:14.879212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:14.879230 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:14.879267 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:14.879281 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:14.879343 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:14.879563 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 09:19:14.879641 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:19:14.881232 14995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:14.869504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:43655 (local address 127.17.16.66:52475)
0811 09:19:14.870453 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:14.870493 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:14.872177 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:14.872875 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:14.872883 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:14.873002 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:14.873725 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:14.873748 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:14.875234 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:14.875239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:14.877140 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:14.877158 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:14.877898 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:14.877904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:14.878163 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:14.878865 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:19:14.878869 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:14.878919 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:19:14.879011 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:14.879363 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:14.879372 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:14.879380 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:19:14.879535 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 09:19:14.879903 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":288,"threads_started":1}
I20260811 09:19:14.882498 14991 heartbeater.cc:347] Connected to a master server at 127.17.16.126:43655
I20260811 09:19:14.882789 14991 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:14.883499 14991 heartbeater.cc:511] Master 127.17.16.126:43655 requested a full tablet report, sending...
I20260811 09:19:14.884826 14491 ts_manager.cc:194] Registered new tserver with Master: bc0b4c1a223b420a837d8d34087100ef (127.17.16.66:38847)
I20260811 09:19:14.885620 14491 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.66:52475
W20260811 09:19:15.216542 14997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:15.216809 14997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:15.216950 14997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:15.224200 14997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:15.224298 14997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:15.224442 14997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:19:15.233578 14997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:43655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:15.234862 14997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:15.236660 14997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:15.249313 15003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:15.249281 15004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:15.249372 15007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:15.249965 14997 server_base.cc:1034] running on GCE node
I20260811 09:19:15.250602 14997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:15.251505 14997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:15.252722 14997 hybrid_clock.cc:648] HybridClock initialized: now 1786439955252666 us; error 49 us; skew 500 ppm
I20260811 09:19:15.253065 14997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:15.255162 14997 webserver.cc:460] Webserver started at http://127.17.16.67:41001/ using document root <none> and password file <none>
I20260811 09:19:15.255921 14997 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:15.256021 14997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:15.256327 14997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:15.259198 14997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "ee3db66eed304e2ba507b93393a72d57"
format_stamp: "Formatted at 2026-08-11 09:19:15 on dist-test-slave-ncp9"
server_key: "389628bdfae03cd576575a2c1364be8c"
server_key_iv: "88e6c4c1e94f3046f5a379e803026754"
server_key_version: "encryptionkey@0"
I20260811 09:19:15.259954 14997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "ee3db66eed304e2ba507b93393a72d57"
format_stamp: "Formatted at 2026-08-11 09:19:15 on dist-test-slave-ncp9"
server_key: "389628bdfae03cd576575a2c1364be8c"
server_key_iv: "88e6c4c1e94f3046f5a379e803026754"
server_key_version: "encryptionkey@0"
I20260811 09:19:15.264797 14997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:19:15.268095 15015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:15.269375 14997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:19:15.269541 14997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "ee3db66eed304e2ba507b93393a72d57"
format_stamp: "Formatted at 2026-08-11 09:19:15 on dist-test-slave-ncp9"
server_key: "389628bdfae03cd576575a2c1364be8c"
server_key_iv: "88e6c4c1e94f3046f5a379e803026754"
server_key_version: "encryptionkey@0"
I20260811 09:19:15.269739 14997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:15.290545 14997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:15.291440 14997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:15.291772 14997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:15.292806 14997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:15.294320 14997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:15.294443 14997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:15.294553 14997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:15.294623 14997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:15.324223 14997 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:37697
I20260811 09:19:15.324285 15195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:37697 every 8 connection(s)
I20260811 09:19:15.325628 14997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:19:15.327155 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 14997
I20260811 09:19:15.327304 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:19:15.327723 17473 external_mini_cluster.cc:1428] Setting key 12bc0297d0ca16ff5c7d7006394e94a6
I20260811 09:19:15.338389 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.329384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:52757 (local address 127.17.16.126:43655)
0811 09:19:15.329605 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:15.329611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:15.331144 (+ 1533us) server_negotiation.cc:407] Connection header received
0811 09:19:15.332794 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:15.332797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:15.332863 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:15.332968 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:15.333922 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.334645 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.336082 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.336464 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.336548 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.337752 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:15.337757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:15.337760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:15.337777 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:15.337809 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:15.337822 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:15.337887 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:15.338051 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 09:19:15.338129 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:19:15.339300 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.330031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:43655 (local address 127.17.16.67:52757)
0811 09:19:15.330984 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:15.331036 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:15.332584 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:15.333155 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:15.333163 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:15.333243 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:15.333797 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.333813 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.334753 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.334756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.335967 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.335978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.336622 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.336627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.336818 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.337462 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:19:15.337465 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:15.337502 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:19:15.337585 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:15.337891 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:15.337896 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:15.337900 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 09:19:15.338033 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 09:19:15.338316 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":314,"threads_started":1}
I20260811 09:19:15.340477 15197 heartbeater.cc:347] Connected to a master server at 127.17.16.126:43655
I20260811 09:19:15.340756 15197 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:15.341604 15197 heartbeater.cc:511] Master 127.17.16.126:43655 requested a full tablet report, sending...
I20260811 09:19:15.343046 14491 ts_manager.cc:194] Registered new tserver with Master: ee3db66eed304e2ba507b93393a72d57 (127.17.16.67:37697)
I20260811 09:19:15.343921 14491 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.67:52757
I20260811 09:19:15.355036 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:19:15.364486 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.359309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43974 (local address 127.17.16.126:43655)
0811 09:19:15.359472 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:15.359476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:15.359895 (+ 419us) server_negotiation.cc:407] Connection header received
0811 09:19:15.360030 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:15.360034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:15.360103 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:15.360188 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:15.360598 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.361319 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.362601 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.362971 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.363056 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.363960 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:15.363965 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:15.363968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:15.363985 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:15.364017 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:15.364030 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:15.364092 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:15.364195 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 09:19:15.364270 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:19:15.368062 14491 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:19:15.374732 14491 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:19:15.398206 15226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.387199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57594 (local address 127.17.16.67:37697)
0811 09:19:15.387827 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:15.387836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:15.388385 (+ 549us) server_negotiation.cc:407] Connection header received
0811 09:19:15.388681 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:15.388690 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:15.388970 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:15.389133 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:15.390154 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.391884 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.394876 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.396411 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.396520 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.396836 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:15.397577 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 09:19:15.397828 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":310,"threads_started":1}
I20260811 09:19:15.399272 15227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.386606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:37697 (local address 127.0.0.1:57594)
0811 09:19:15.388272 (+ 1666us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:15.388290 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:15.388489 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:15.389479 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:15.389487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:15.389520 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:15.389918 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.389930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.392094 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.392100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.394625 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.394651 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.397053 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.397060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.397298 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.397350 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 09:19:15.398738 (+ 1388us) client_negotiation.cc:240] Negotiation successful
0811 09:19:15.398959 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1536,"threads_started":1}
I20260811 09:19:15.403905 15096 tablet_service.cc:1467] Processing CreateTablet for tablet a0e54b4f6e574692a99764301ae90c12 (DEFAULT_TABLE table=test-table [id=e8c5f30b1fff482b8557ad3838ca1e77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:15.405418 15096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e54b4f6e574692a99764301ae90c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:15.407172 15225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.387580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:38847 (local address 127.0.0.1:38306)
0811 09:19:15.399359 (+ 11779us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:15.399379 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:15.399488 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:15.400666 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:15.400672 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:15.400704 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:15.401051 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.401064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.402823 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.402828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.404854 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.404869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.406458 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.406465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.406624 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.406657 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:19:15.406736 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 09:19:15.406871 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11657,"thread_start_us":690,"threads_started":1}
I20260811 09:19:15.407297 15228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.388082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38306 (local address 127.17.16.66:38847)
0811 09:19:15.400025 (+ 11943us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:15.400033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:15.400094 (+ 61us) server_negotiation.cc:407] Connection header received
0811 09:19:15.400209 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:15.400217 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:15.400465 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:15.400666 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:15.401228 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.402685 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.405078 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.406276 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.406395 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.406632 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:15.406810 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 09:19:15.406970 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11771,"thread_start_us":11596,"threads_started":1}
I20260811 09:19:15.408718 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.385915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:40053 (local address 127.0.0.1:35868)
0811 09:19:15.386668 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:15.386710 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:15.387217 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:15.401919 (+ 14702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:15.401924 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:15.401952 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:15.402364 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.402376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.403978 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.403982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.405678 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.405694 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.408043 (+ 2349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.408050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.408237 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.408270 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:19:15.408338 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 09:19:15.408465 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":2,"thread_start_us":836,"threads_started":1}
I20260811 09:19:15.409646 15221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.386011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35868 (local address 127.17.16.65:40053)
0811 09:19:15.401333 (+ 15322us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:15.401343 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:15.401373 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:19:15.401471 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:15.401478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:15.401712 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:15.402566 (+ 854us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:15.402846 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.403840 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.405882 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.407830 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.409047 (+ 1217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.409222 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:15.409283 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 09:19:15.409397 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15131,"thread_start_us":16770,"threads_started":1}
I20260811 09:19:15.411268 14880 tablet_service.cc:1467] Processing CreateTablet for tablet a0e54b4f6e574692a99764301ae90c12 (DEFAULT_TABLE table=test-table [id=e8c5f30b1fff482b8557ad3838ca1e77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:15.412709 14880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e54b4f6e574692a99764301ae90c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:15.412390 14682 tablet_service.cc:1467] Processing CreateTablet for tablet a0e54b4f6e574692a99764301ae90c12 (DEFAULT_TABLE table=test-table [id=e8c5f30b1fff482b8557ad3838ca1e77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:15.413619 14682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e54b4f6e574692a99764301ae90c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:15.425652 15230 tablet_bootstrap.cc:492] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: Bootstrap starting.
I20260811 09:19:15.428838 15233 tablet_bootstrap.cc:492] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: Bootstrap starting.
I20260811 09:19:15.429009 15230 tablet_bootstrap.cc:654] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:15.430317 15234 tablet_bootstrap.cc:492] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: Bootstrap starting.
I20260811 09:19:15.430366 15230 log.cc:824] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:15.432255 15233 tablet_bootstrap.cc:654] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:15.433491 15234 tablet_bootstrap.cc:654] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:15.433516 15233 log.cc:824] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:15.434197 15230 tablet_bootstrap.cc:492] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: No bootstrap required, opened a new log
I20260811 09:19:15.434599 15234 log.cc:824] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:15.434947 15230 ts_tablet_manager.cc:1397] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260811 09:19:15.436710 15233 tablet_bootstrap.cc:492] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: No bootstrap required, opened a new log
I20260811 09:19:15.437013 15233 ts_tablet_manager.cc:1397] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260811 09:19:15.437592 15234 tablet_bootstrap.cc:492] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: No bootstrap required, opened a new log
I20260811 09:19:15.437858 15234 ts_tablet_manager.cc:1397] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:19:15.438391 14785 heartbeater.cc:503] Master 127.17.16.126:43655 was elected leader, sending a full tablet report...
I20260811 09:19:15.447429 15233 raft_consensus.cc:348] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 } } peers { permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 } } peers { permanent_uuid: "ee3db66eed304e2ba507b93393a72d57" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 37697 } }
I20260811 09:19:15.448064 15233 raft_consensus.cc:374] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:15.448217 15233 raft_consensus.cc:729] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ddfa8a2c9b34abbb238bcae67db882a, State: Initialized, Role: FOLLOWER
I20260811 09:19:15.448004 15234 raft_consensus.cc:348] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 } } peers { permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 } } peers { permanent_uuid: "ee3db66eed304e2ba507b93393a72d57" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 37697 } }
I20260811 09:19:15.448065 15230 raft_consensus.cc:348] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 } } peers { permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 } } peers { permanent_uuid: "ee3db66eed304e2ba507b93393a72d57" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 37697 } }
I20260811 09:19:15.448523 15234 raft_consensus.cc:374] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:15.448547 15230 raft_consensus.cc:374] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:15.448637 15230 raft_consensus.cc:729] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3db66eed304e2ba507b93393a72d57, State: Initialized, Role: FOLLOWER
I20260811 09:19:15.448680 15234 raft_consensus.cc:729] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc0b4c1a223b420a837d8d34087100ef, State: Initialized, Role: FOLLOWER
I20260811 09:19:15.449108 15233 consensus_queue.cc:260] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 } } peers { permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 } } peers { permanent_uuid: "ee3db66eed304e2ba507b93393a72d57" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 37697 } }
I20260811 09:19:15.449398 15230 consensus_queue.cc:260] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 } } peers { permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 } } peers { permanent_uuid: "ee3db66eed304e2ba507b93393a72d57" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 37697 } }
I20260811 09:19:15.449537 15234 consensus_queue.cc:260] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 } } peers { permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 } } peers { permanent_uuid: "ee3db66eed304e2ba507b93393a72d57" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 37697 } }
I20260811 09:19:15.454545 14991 heartbeater.cc:503] Master 127.17.16.126:43655 was elected leader, sending a full tablet report...
I20260811 09:19:15.455039 15197 heartbeater.cc:503] Master 127.17.16.126:43655 was elected leader, sending a full tablet report...
I20260811 09:19:15.455408 15234 ts_tablet_manager.cc:1428] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: Time spent starting tablet: real 0.017s user 0.018s sys 0.000s
I20260811 09:19:15.455523 15233 ts_tablet_manager.cc:1428] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: Time spent starting tablet: real 0.018s user 0.010s sys 0.007s
I20260811 09:19:15.456447 15230 ts_tablet_manager.cc:1428] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: Time spent starting tablet: real 0.021s user 0.009s sys 0.007s
W20260811 09:19:15.577291 15198 tablet.cc:2369] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:19:15.592065 15241 raft_consensus.cc:482] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:19:15.592406 15241 raft_consensus.cc:504] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 } } peers { permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 } } peers { permanent_uuid: "ee3db66eed304e2ba507b93393a72d57" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 37697 } }
I20260811 09:19:15.594316 15241 leader_election.cc:288] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ddfa8a2c9b34abbb238bcae67db882a (127.17.16.65:40053), bc0b4c1a223b420a837d8d34087100ef (127.17.16.66:38847)
I20260811 09:19:15.600106 15221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.594813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:49007 (local address 127.17.16.65:40053)
0811 09:19:15.595000 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:15.595006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:15.595028 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:19:15.595199 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:15.595204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:15.595282 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:15.595376 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:15.596036 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.596728 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.598445 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.599230 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.599375 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.599481 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:15.599721 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 09:19:15.599846 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:19:15.600107 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.594694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:40053 (local address 127.17.16.67:49007)
0811 09:19:15.594894 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:15.594916 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:15.595053 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:15.595488 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:15.595495 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:15.595537 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:15.595809 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.595824 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.596878 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.596884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.598277 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.598290 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.599381 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.599387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.599564 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.599588 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:19:15.599669 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 09:19:15.599803 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 09:19:15.601166 15245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.595221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:38847 (local address 127.17.16.67:60545)
0811 09:19:15.596086 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:15.596100 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:15.596226 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:15.596535 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:15.596540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:15.596564 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:15.596882 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.596924 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.597916 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.597921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.599247 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:15.599260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.600521 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.600527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:15.600687 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.600719 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:19:15.600807 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 09:19:15.600939 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":575,"threads_started":1}
I20260811 09:19:15.601269 15228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.595436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:60545 (local address 127.17.16.66:38847)
0811 09:19:15.595642 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:15.595647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:15.596180 (+ 533us) server_negotiation.cc:407] Connection header received
0811 09:19:15.596352 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:15.596357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:15.596430 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:15.596503 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:15.597086 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.597773 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.599426 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.600255 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.600399 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.600461 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:15.600841 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 09:19:15.600994 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:19:15.601359 14710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e54b4f6e574692a99764301ae90c12" candidate_uuid: "ee3db66eed304e2ba507b93393a72d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" is_pre_election: true
I20260811 09:19:15.601902 14710 raft_consensus.cc:2454] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3db66eed304e2ba507b93393a72d57 in term 0.
I20260811 09:19:15.602108 14910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e54b4f6e574692a99764301ae90c12" candidate_uuid: "ee3db66eed304e2ba507b93393a72d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc0b4c1a223b420a837d8d34087100ef" is_pre_election: true
I20260811 09:19:15.602576 15017 leader_election.cc:302] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ddfa8a2c9b34abbb238bcae67db882a, ee3db66eed304e2ba507b93393a72d57; no voters:
I20260811 09:19:15.602655 14910 raft_consensus.cc:2454] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3db66eed304e2ba507b93393a72d57 in term 0.
I20260811 09:19:15.602988 15241 raft_consensus.cc:2781] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:19:15.603129 15241 raft_consensus.cc:482] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:19:15.603240 15241 raft_consensus.cc:3037] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:15.605472 15241 raft_consensus.cc:504] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 } } peers { permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 } } peers { permanent_uuid: "ee3db66eed304e2ba507b93393a72d57" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 37697 } }
I20260811 09:19:15.606014 15241 leader_election.cc:288] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [CANDIDATE]: Term 1 election: Requested vote from peers 8ddfa8a2c9b34abbb238bcae67db882a (127.17.16.65:40053), bc0b4c1a223b420a837d8d34087100ef (127.17.16.66:38847)
I20260811 09:19:15.606482 14710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e54b4f6e574692a99764301ae90c12" candidate_uuid: "ee3db66eed304e2ba507b93393a72d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ddfa8a2c9b34abbb238bcae67db882a"
I20260811 09:19:15.606576 14910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e54b4f6e574692a99764301ae90c12" candidate_uuid: "ee3db66eed304e2ba507b93393a72d57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc0b4c1a223b420a837d8d34087100ef"
I20260811 09:19:15.606676 14710 raft_consensus.cc:3037] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:15.606770 14910 raft_consensus.cc:3037] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:15.608915 14710 raft_consensus.cc:2454] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3db66eed304e2ba507b93393a72d57 in term 1.
I20260811 09:19:15.609046 14910 raft_consensus.cc:2454] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3db66eed304e2ba507b93393a72d57 in term 1.
I20260811 09:19:15.609330 15017 leader_election.cc:302] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ddfa8a2c9b34abbb238bcae67db882a, ee3db66eed304e2ba507b93393a72d57; no voters:
I20260811 09:19:15.609611 15241 raft_consensus.cc:2781] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:15.610042 15241 raft_consensus.cc:686] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 1 LEADER]: Becoming Leader. State: Replica: ee3db66eed304e2ba507b93393a72d57, State: Running, Role: LEADER
I20260811 09:19:15.610487 15241 consensus_queue.cc:237] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 } } peers { permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 } } peers { permanent_uuid: "ee3db66eed304e2ba507b93393a72d57" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 37697 } }
I20260811 09:19:15.615106 14490 catalog_manager.cc:5495] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee3db66eed304e2ba507b93393a72d57 (127.17.16.67). New cstate: current_term: 1 leader_uuid: "ee3db66eed304e2ba507b93393a72d57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee3db66eed304e2ba507b93393a72d57" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 37697 } health_report { overall_health: HEALTHY } } }
W20260811 09:19:15.618284 14994 tablet.cc:2369] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:19:15.637368 15226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.632076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57610 (local address 127.17.16.67:37697)
0811 09:19:15.632302 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:15.632308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:15.632328 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:19:15.632422 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:15.632426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:15.632539 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:15.632668 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:19:15.633109 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.633864 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.635226 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.635641 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.635748 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.636263 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:19:15.636774 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:19:15.636925 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:15.637048 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 09:19:15.637130 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 09:19:15.645731 14910 raft_consensus.cc:1261] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef [term 1 FOLLOWER]: Refusing update from remote peer ee3db66eed304e2ba507b93393a72d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:15.645731 14710 raft_consensus.cc:1261] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a [term 1 FOLLOWER]: Refusing update from remote peer ee3db66eed304e2ba507b93393a72d57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:15.646706 15246 consensus_queue.cc:1035] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc0b4c1a223b420a837d8d34087100ef" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 38847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:15.647078 15241 consensus_queue.cc:1035] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ddfa8a2c9b34abbb238bcae67db882a" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:15.660423 15253 mvcc.cc:205] Tried to move back new op lower bound from 7317258058313719808 to 7317258058184491008. Current Snapshot: MvccSnapshot[applied={T|T < 7317258058313719808}]
I20260811 09:19:15.662350 15255 mvcc.cc:205] Tried to move back new op lower bound from 7317258058313719808 to 7317258058184491008. Current Snapshot: MvccSnapshot[applied={T|T < 7317258058313719808}]
I20260811 09:19:15.672358 14489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:15.672655 14489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:15.676345 14489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcbfb41672f24c54926d4b0d18fecfd0: Sending DeleteTablet for 3 replicas of tablet a0e54b4f6e574692a99764301ae90c12
I20260811 09:19:15.677637 14682 tablet_service.cc:1514] Processing DeleteTablet for tablet a0e54b4f6e574692a99764301ae90c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:15 UTC) from {username='slave'} at 127.0.0.1:35868
I20260811 09:19:15.677685 14880 tablet_service.cc:1514] Processing DeleteTablet for tablet a0e54b4f6e574692a99764301ae90c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:15 UTC) from {username='slave'} at 127.0.0.1:38306
I20260811 09:19:15.678459 15269 tablet_replica.cc:330] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: stopping tablet replica
I20260811 09:19:15.678576 15096 tablet_service.cc:1514] Processing DeleteTablet for tablet a0e54b4f6e574692a99764301ae90c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:15 UTC) from {username='slave'} at 127.0.0.1:57594
I20260811 09:19:15.679005 15269 raft_consensus.cc:2229] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:15.679563 15269 raft_consensus.cc:2258] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:15.679603 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 14591
I20260811 09:19:15.681857 15269 ts_tablet_manager.cc:1905] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:15.689280 15269 ts_tablet_manager.cc:1918] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:15.689543 15269 log.cc:1196] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/a0e54b4f6e574692a99764301ae90c12
I20260811 09:19:15.690153 15269 ts_tablet_manager.cc:1939] T a0e54b4f6e574692a99764301ae90c12 P 8ddfa8a2c9b34abbb238bcae67db882a: Deleting consensus metadata
I20260811 09:19:15.692004 14473 catalog_manager.cc:4841] TS 8ddfa8a2c9b34abbb238bcae67db882a (127.17.16.65:40053): tablet a0e54b4f6e574692a99764301ae90c12 (table test-table [id=e8c5f30b1fff482b8557ad3838ca1e77]) successfully deleted
I20260811 09:19:15.693037 15273 tablet_replica.cc:330] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: stopping tablet replica
I20260811 09:19:15.693470 15273 raft_consensus.cc:2229] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:19:15.694118 15273 raft_consensus.cc:2258] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:15.694976 15271 tablet_replica.cc:330] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: stopping tablet replica
I20260811 09:19:15.695384 15271 raft_consensus.cc:2229] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:15.695955 15271 raft_consensus.cc:2258] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:15.698009 15271 ts_tablet_manager.cc:1905] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:15.699867 15221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.680860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.17.16.65:40053)
0811 09:19:15.681047 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:15.681053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:15.690824 (+ 9771us) server_negotiation.cc:407] Connection header received
0811 09:19:15.691280 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:15.691285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:15.691379 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:15.691451 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:15.692572 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.693908 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.695942 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.696571 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.696699 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.698222 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:15.698234 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:15.698250 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:15.698978 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:15.699036 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:15.699060 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:15.699142 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:15.699387 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 09:19:15.699514 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:19:15.701088 14780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:15.701273 15273 ts_tablet_manager.cc:1905] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:15.710183 15271 ts_tablet_manager.cc:1918] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:15.710350 15271 log.cc:1196] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/a0e54b4f6e574692a99764301ae90c12
I20260811 09:19:15.710780 15271 ts_tablet_manager.cc:1939] T a0e54b4f6e574692a99764301ae90c12 P bc0b4c1a223b420a837d8d34087100ef: Deleting consensus metadata
I20260811 09:19:15.712390 14473 catalog_manager.cc:4841] TS bc0b4c1a223b420a837d8d34087100ef (127.17.16.66:38847): tablet a0e54b4f6e574692a99764301ae90c12 (table test-table [id=e8c5f30b1fff482b8557ad3838ca1e77]) successfully deleted
I20260811 09:19:15.715111 15273 ts_tablet_manager.cc:1918] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:15.715296 15273 log.cc:1196] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/a0e54b4f6e574692a99764301ae90c12
I20260811 09:19:15.715775 15273 ts_tablet_manager.cc:1939] T a0e54b4f6e574692a99764301ae90c12 P ee3db66eed304e2ba507b93393a72d57: Deleting consensus metadata
I20260811 09:19:15.717357 14474 catalog_manager.cc:4841] TS ee3db66eed304e2ba507b93393a72d57 (127.17.16.67:37697): tablet a0e54b4f6e574692a99764301ae90c12 (table test-table [id=e8c5f30b1fff482b8557ad3838ca1e77]) successfully deleted
I20260811 09:19:15.850087 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 14591
I20260811 09:19:15.874253 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 14789
I20260811 09:19:15.884516 15228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:15.875714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.17.16.66:38847)
0811 09:19:15.875898 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:15.875905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:15.875925 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:19:15.875988 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:15.875993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:15.876134 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:15.876240 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:15.876869 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.878090 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.880210 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:15.880881 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:15.881125 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:15.882636 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:15.882647 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:15.882667 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:15.883469 (+ 802us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:15.883543 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:15.883570 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:15.883680 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:15.883853 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 09:19:15.883968 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:19:15.885267 14985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:15.974009 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 14789
I20260811 09:19:16.007648 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 14997
I20260811 09:19:16.015321 15226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:16.008447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57616 (local address 127.17.16.67:37697)
0811 09:19:16.008655 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:16.008661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:16.008681 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:19:16.008880 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:16.008885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:16.009019 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:16.009113 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:16.009540 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:16.010333 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:16.011635 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:16.012090 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:16.012203 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:16.013401 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:16.013414 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:16.013434 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:16.014388 (+ 954us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:16.014472 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:16.014505 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:16.014632 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:16.014796 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 09:19:16.014906 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:19:16.016177 15188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:16.136525 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 14997
I20260811 09:19:16.159003 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 14446
I20260811 09:19:16.160094 14543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:16.268174 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 14446
2026-08-11T09:19:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2944 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:16 dist-test-slave-ncp9 krb5kdc[15286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:16 dist-test-slave-ncp9 krb5kdc[15286](info): set up 2 sockets
Aug 11 09:19:16 dist-test-slave-ncp9 krb5kdc[15286](info): commencing operation
krb5kdc: starting...
W20260811 09:19:17.534092 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:17 dist-test-slave-ncp9 krb5kdc[15286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:19:17.849000 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:38529
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:38529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:18.208552 15306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:18.208838 15306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:18.208966 15306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:18.216164 15306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:18.216285 15306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:18.216338 15306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:18.216395 15306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:18.216455 15306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:18.226572 15306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:38529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:38529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:18.228016 15306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:18.229336 15306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:18.240783 15313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:18.240826 15318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:18.240753 15314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:18.241503 15306 server_base.cc:1034] running on GCE node
I20260811 09:19:18.242348 15306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:18.243786 15306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:18.245000 15306 hybrid_clock.cc:648] HybridClock initialized: now 1786439958244938 us; error 44 us; skew 500 ppm
I20260811 09:19:18.245352 15306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:18 dist-test-slave-ncp9 krb5kdc[15286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439958, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:18.248432 15306 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:19:18.249902 15306 webserver.cc:460] Webserver started at http://127.17.16.126:41657/ using document root <none> and password file <none>
I20260811 09:19:18.250577 15306 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:18.250697 15306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:18.251000 15306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:18.254047 15306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "f3c445dae6d14fe18069dde4d7239a51"
format_stamp: "Formatted at 2026-08-11 09:19:18 on dist-test-slave-ncp9"
server_key: "21138607a137c72469f0312b7f722e5d"
server_key_iv: "b355c12ce3d0213b4db8e551fee81c93"
server_key_version: "encryptionkey@0"
I20260811 09:19:18.254773 15306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "f3c445dae6d14fe18069dde4d7239a51"
format_stamp: "Formatted at 2026-08-11 09:19:18 on dist-test-slave-ncp9"
server_key: "21138607a137c72469f0312b7f722e5d"
server_key_iv: "b355c12ce3d0213b4db8e551fee81c93"
server_key_version: "encryptionkey@0"
I20260811 09:19:18.259884 15306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 09:19:18.263243 15329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:18.264605 15306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:19:18.264792 15306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "f3c445dae6d14fe18069dde4d7239a51"
format_stamp: "Formatted at 2026-08-11 09:19:18 on dist-test-slave-ncp9"
server_key: "21138607a137c72469f0312b7f722e5d"
server_key_iv: "b355c12ce3d0213b4db8e551fee81c93"
server_key_version: "encryptionkey@0"
I20260811 09:19:18.265033 15306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:18.284416 15306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:18.288349 15306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:18.288637 15306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:18.306008 15306 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:38529
I20260811 09:19:18.306015 15419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:38529 every 8 connection(s)
I20260811 09:19:18.307385 15306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:18.309093 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 15306
I20260811 09:19:18.309273 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:18.309686 17473 external_mini_cluster.cc:1428] Setting key 0b39ac2d8b1ded0e43da1b0155580477
I20260811 09:19:18.311512 15421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:19:18 dist-test-slave-ncp9 krb5kdc[15286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:18.325358 15421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51: Bootstrap starting.
I20260811 09:19:18.328765 15421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:18.329815 15421 log.cc:824] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:18.333249 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:18.311504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39294 (local address 127.17.16.126:38529)
0811 09:19:18.312304 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:18.312318 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:18.312370 (+ 52us) server_negotiation.cc:407] Connection header received
0811 09:19:18.314543 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:18.314570 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:18.315083 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:18.315517 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:18.316968 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:18.318956 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:18.320329 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:18.321361 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:18.321550 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:18.325697 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:18.325706 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:18.325720 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:18.325757 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:18.328739 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:18.329461 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:18.329467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:18.329474 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:18.329641 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:18.330027 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:18.330032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:18.330035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:18.330724 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:18.330971 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:18.331481 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 09:19:18.331910 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":317,"threads_started":1}
I20260811 09:19:18.333314 15421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51: No bootstrap required, opened a new log
I20260811 09:19:18.341409 15421 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c445dae6d14fe18069dde4d7239a51" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38529 } }
I20260811 09:19:18.341784 15421 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:18.341866 15421 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c445dae6d14fe18069dde4d7239a51, State: Initialized, Role: FOLLOWER
I20260811 09:19:18.342412 15421 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c445dae6d14fe18069dde4d7239a51" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38529 } }
I20260811 09:19:18.342635 15421 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:18.342738 15421 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:18.342871 15421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:18.345022 15421 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c445dae6d14fe18069dde4d7239a51" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38529 } }
I20260811 09:19:18.345469 15421 leader_election.cc:302] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3c445dae6d14fe18069dde4d7239a51; no voters:
I20260811 09:19:18.346031 15421 leader_election.cc:288] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:18.346226 15432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:18.347259 15432 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [term 1 LEADER]: Becoming Leader. State: Replica: f3c445dae6d14fe18069dde4d7239a51, State: Running, Role: LEADER
I20260811 09:19:18.347764 15432 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c445dae6d14fe18069dde4d7239a51" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38529 } }
I20260811 09:19:18.348073 15421 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:18.350325 15434 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3c445dae6d14fe18069dde4d7239a51. Latest consensus state: current_term: 1 leader_uuid: "f3c445dae6d14fe18069dde4d7239a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c445dae6d14fe18069dde4d7239a51" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38529 } } }
I20260811 09:19:18.350668 15434 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:18.350950 15433 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3c445dae6d14fe18069dde4d7239a51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c445dae6d14fe18069dde4d7239a51" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38529 } } }
I20260811 09:19:18.351274 15433 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:18.353533 15446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:18.358958 15446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:18.367987 15446 catalog_manager.cc:1347] Generated new cluster ID: 724c38d10ecc479ba7405bf6c0f8de49
I20260811 09:19:18.368121 15446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:18.383832 15446 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:18.385744 15446 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:18.402508 15446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51: Generated new TSK 0
I20260811 09:19:18.403693 15446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:19:18.484377 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38529
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:18.827004 15470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:18.827308 15470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:18.827432 15470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:18.834748 15470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:18.834863 15470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:18.835021 15470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:19:18.845224 15470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:18.846621 15470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:18.848007 15470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:18.861217 15476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:18.861261 15477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:18.861264 15480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:18.862358 15470 server_base.cc:1034] running on GCE node
I20260811 09:19:18.862989 15470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:18.863965 15470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:18.865172 15470 hybrid_clock.cc:648] HybridClock initialized: now 1786439958865125 us; error 32 us; skew 500 ppm
I20260811 09:19:18.865486 15470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:18 dist-test-slave-ncp9 krb5kdc[15286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439958, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:18.868780 15470 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:19:18.870373 15470 webserver.cc:460] Webserver started at http://127.17.16.65:34511/ using document root <none> and password file <none>
I20260811 09:19:18.871081 15470 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:18.871203 15470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:18.871517 15470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:18.874648 15470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "49096c43331f48da9a44cab454617e72"
format_stamp: "Formatted at 2026-08-11 09:19:18 on dist-test-slave-ncp9"
server_key: "eef9a349d05d8a9317456917e37658b1"
server_key_iv: "3a3ec40eab319767636b047a954c8797"
server_key_version: "encryptionkey@0"
I20260811 09:19:18.875439 15470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "49096c43331f48da9a44cab454617e72"
format_stamp: "Formatted at 2026-08-11 09:19:18 on dist-test-slave-ncp9"
server_key: "eef9a349d05d8a9317456917e37658b1"
server_key_iv: "3a3ec40eab319767636b047a954c8797"
server_key_version: "encryptionkey@0"
I20260811 09:19:18.880990 15470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 09:19:18.884519 15494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:18.885916 15470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:19:18.886080 15470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "49096c43331f48da9a44cab454617e72"
format_stamp: "Formatted at 2026-08-11 09:19:18 on dist-test-slave-ncp9"
server_key: "eef9a349d05d8a9317456917e37658b1"
server_key_iv: "3a3ec40eab319767636b047a954c8797"
server_key_version: "encryptionkey@0"
I20260811 09:19:18.886274 15470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:18.918955 15470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:18.923041 15470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:18.923318 15470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:18.924304 15470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:18.925717 15470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:18.925798 15470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:18.925916 15470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:18.925978 15470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:18.957005 15470 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:35695
I20260811 09:19:18.957077 15677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:35695 every 8 connection(s)
I20260811 09:19:18.958320 15470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:19:18.967823 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 15470
I20260811 09:19:18.968003 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:19:18.968504 17473 external_mini_cluster.cc:1428] Setting key c4d38963fa77a0b93d6f433dc95c729b
Aug 11 09:19:18 dist-test-slave-ncp9 krb5kdc[15286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439958, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:18.977685 15681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:18.960754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:52157 (local address 127.17.16.126:38529)
0811 09:19:18.961230 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:18.961235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:18.962061 (+ 826us) server_negotiation.cc:407] Connection header received
0811 09:19:18.963911 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:18.963916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:18.964000 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:18.964100 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:18.965647 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:18.966359 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:18.967913 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:18.968485 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:18.968604 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:18.972255 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:18.972260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:18.972263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:18.972296 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:18.974620 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:18.975321 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:18.975325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:18.975327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:18.975451 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:18.975882 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:18.975885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:18.975887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:18.976163 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:18.976297 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:18.977111 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 09:19:18.977343 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":262,"threads_started":1}
I20260811 09:19:18.978629 15682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:18.961225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:38529 (local address 127.17.16.65:52157)
0811 09:19:18.961888 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:18.961926 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:18.963635 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:18.964351 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:18.964360 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:18.964867 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:18.965473 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:18.965491 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:18.966514 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:18.966518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:18.967764 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:18.967777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:18.968714 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:18.968721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:18.969047 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:18.969739 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:19:18.969767 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:18.972039 (+ 2272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:18.974755 (+ 2716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:18.974764 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:18.974782 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:18.975195 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:18.975556 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:18.975561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:18.975565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:18.975765 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:18.976290 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:18.976298 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:18.976669 (+ 371us) client_negotiation.cc:769] Sending connection context
0811 09:19:18.976941 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 09:19:18.977480 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":267,"threads_started":1}
I20260811 09:19:18.980132 15678 heartbeater.cc:347] Connected to a master server at 127.17.16.126:38529
I20260811 09:19:18.980463 15678 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:18.981333 15678 heartbeater.cc:511] Master 127.17.16.126:38529 requested a full tablet report, sending...
I20260811 09:19:18.983624 15360 ts_manager.cc:194] Registered new tserver with Master: 49096c43331f48da9a44cab454617e72 (127.17.16.65:35695)
I20260811 09:19:18.985138 15360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:52157
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:19:19.037495 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38529
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:19.383399 15687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:19.383675 15687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:19.383844 15687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:19.391139 15687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:19.391245 15687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:19.391392 15687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:19:19.401230 15687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:19.402520 15687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:19.403936 15687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:19.416951 15697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:19.416920 15699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:19.416848 15705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:19.417752 15687 server_base.cc:1034] running on GCE node
I20260811 09:19:19.418345 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:19.419298 15687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:19.420511 15687 hybrid_clock.cc:648] HybridClock initialized: now 1786439959420461 us; error 32 us; skew 500 ppm
I20260811 09:19:19.420823 15687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:19 dist-test-slave-ncp9 krb5kdc[15286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439959, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:19.423889 15687 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:19:19.425315 15687 webserver.cc:460] Webserver started at http://127.17.16.66:42423/ using document root <none> and password file <none>
I20260811 09:19:19.425953 15687 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:19.426061 15687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:19.426334 15687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:19.429306 15687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "13950d04616c4f7da198df24972f5266"
format_stamp: "Formatted at 2026-08-11 09:19:19 on dist-test-slave-ncp9"
server_key: "cd1ee2fe5cfd6019105059c6743aec40"
server_key_iv: "b951490e69646c7d21c9774e65f9aab6"
server_key_version: "encryptionkey@0"
I20260811 09:19:19.430052 15687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "13950d04616c4f7da198df24972f5266"
format_stamp: "Formatted at 2026-08-11 09:19:19 on dist-test-slave-ncp9"
server_key: "cd1ee2fe5cfd6019105059c6743aec40"
server_key_iv: "b951490e69646c7d21c9774e65f9aab6"
server_key_version: "encryptionkey@0"
I20260811 09:19:19.435187 15687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:19:19.438759 15719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:19.440093 15687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:19:19.440238 15687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "13950d04616c4f7da198df24972f5266"
format_stamp: "Formatted at 2026-08-11 09:19:19 on dist-test-slave-ncp9"
server_key: "cd1ee2fe5cfd6019105059c6743aec40"
server_key_iv: "b951490e69646c7d21c9774e65f9aab6"
server_key_version: "encryptionkey@0"
I20260811 09:19:19.440423 15687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:19.462474 15687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:19.466566 15687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:19.466866 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:19.467864 15687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:19.469380 15687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:19.469465 15687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:19.469592 15687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:19.469640 15687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:19.501259 15687 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:45533
I20260811 09:19:19.501302 15904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:45533 every 8 connection(s)
I20260811 09:19:19.502557 15687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:19:19.509600 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 15687
I20260811 09:19:19.509799 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:19:19.510362 17473 external_mini_cluster.cc:1428] Setting key e734c8d476d74a333a7a73ec5e10c66a
Aug 11 09:19:19 dist-test-slave-ncp9 krb5kdc[15286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439959, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:19.523449 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:19.505181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:59525 (local address 127.17.16.126:38529)
0811 09:19:19.505633 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:19.505638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:19.506626 (+ 988us) server_negotiation.cc:407] Connection header received
0811 09:19:19.508400 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:19.508405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:19.508476 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:19.508568 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:19.510437 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:19.511193 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:19.513111 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:19.513572 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:19.513677 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:19.517720 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:19.517725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:19.517729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:19.517762 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:19.520292 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:19.521188 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:19.521192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:19.521195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:19.521325 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:19.521778 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:19.521780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:19.521782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:19.522050 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:19.522162 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:19.523008 (+ 846us) server_negotiation.cc:299] Negotiation successful
0811 09:19:19.523184 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":195,"threads_started":1}
I20260811 09:19:19.524837 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:19.505730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:38529 (local address 127.17.16.66:59525)
0811 09:19:19.506439 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:19.506478 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:19.508159 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:19.508789 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:19.508797 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:19.509458 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:19.510253 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:19.510279 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:19.511347 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:19.511352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:19.512972 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:19.512989 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:19.513811 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:19.513819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:19.514116 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:19.514824 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:19:19.514860 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:19.517449 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:19.520439 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:19.520448 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:19.520467 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:19.521082 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:19.521422 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:19.521427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:19.521430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:19.521683 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:19.522167 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:19.522175 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:19.522641 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 09:19:19.522940 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 09:19:19.523484 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":264,"threads_started":1}
I20260811 09:19:19.526311 15906 heartbeater.cc:347] Connected to a master server at 127.17.16.126:38529
I20260811 09:19:19.526638 15906 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:19.527395 15906 heartbeater.cc:511] Master 127.17.16.126:38529 requested a full tablet report, sending...
I20260811 09:19:19.528851 15360 ts_manager.cc:194] Registered new tserver with Master: 13950d04616c4f7da198df24972f5266 (127.17.16.66:45533)
I20260811 09:19:19.529698 15360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:59525
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:19:19.577849 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38529
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:19.917553 15917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:19.918018 15917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:19.918190 15917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:19.926072 15917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:19.926192 15917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:19.926316 15917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:19:19.936142 15917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:19.937458 15917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:19.938719 15917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:19.951543 15924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:19.952016 15929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:19.952056 15925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:19.952380 15917 server_base.cc:1034] running on GCE node
I20260811 09:19:19.952996 15917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:19.953908 15917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:19.955104 15917 hybrid_clock.cc:648] HybridClock initialized: now 1786439959955064 us; error 40 us; skew 500 ppm
I20260811 09:19:19.955403 15917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:19 dist-test-slave-ncp9 krb5kdc[15286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439959, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:19.958796 15917 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:19:19.960518 15917 webserver.cc:460] Webserver started at http://127.17.16.67:34493/ using document root <none> and password file <none>
I20260811 09:19:19.961386 15917 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:19.961539 15917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:19.961901 15917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:19.964962 15917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7"
format_stamp: "Formatted at 2026-08-11 09:19:19 on dist-test-slave-ncp9"
server_key: "bb689aecac7e7427284f3fe85b6d0516"
server_key_iv: "00dcf061f87d993c40fd05e0a50e3f88"
server_key_version: "encryptionkey@0"
I20260811 09:19:19.965685 15917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7"
format_stamp: "Formatted at 2026-08-11 09:19:19 on dist-test-slave-ncp9"
server_key: "bb689aecac7e7427284f3fe85b6d0516"
server_key_iv: "00dcf061f87d993c40fd05e0a50e3f88"
server_key_version: "encryptionkey@0"
I20260811 09:19:19.970829 15917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260811 09:19:19.974143 15939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:19.975446 15917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:19:19.975602 15917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7"
format_stamp: "Formatted at 2026-08-11 09:19:19 on dist-test-slave-ncp9"
server_key: "bb689aecac7e7427284f3fe85b6d0516"
server_key_iv: "00dcf061f87d993c40fd05e0a50e3f88"
server_key_version: "encryptionkey@0"
I20260811 09:19:19.975787 15917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:19.987960 15678 heartbeater.cc:503] Master 127.17.16.126:38529 was elected leader, sending a full tablet report...
I20260811 09:19:20.006451 15917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:20.010474 15917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:20.010761 15917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:20.011747 15917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:20.013262 15917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:20.013347 15917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:20.013471 15917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:20.013537 15917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:20.044329 15917 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:40047
I20260811 09:19:20.044414 16130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:40047 every 8 connection(s)
I20260811 09:19:20.045854 15917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:19:20.049006 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 15917
I20260811 09:19:20.049217 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:19:20.049825 17473 external_mini_cluster.cc:1428] Setting key 9142b0c686545e0d026515c271472f3c
Aug 11 09:19:20 dist-test-slave-ncp9 krb5kdc[15286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439959, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:20.066104 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.048306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:53549 (local address 127.17.16.126:38529)
0811 09:19:20.048793 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.048797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.049846 (+ 1049us) server_negotiation.cc:407] Connection header received
0811 09:19:20.051957 (+ 2111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.051963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.052049 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.052140 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:20.053849 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.054724 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.056245 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.056653 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.056758 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.060170 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:20.060175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:20.060178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:20.060212 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:20.062774 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:20.063576 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:20.063580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:20.063582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:20.063724 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:20.064250 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:20.064255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:20.064258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:20.064542 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:20.064697 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.065482 (+ 785us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.065717 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":225,"threads_started":1}
I20260811 09:19:20.067034 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.048825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:38529 (local address 127.17.16.67:53549)
0811 09:19:20.049704 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.049741 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:20.051683 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:20.052406 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:20.052418 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:20.053042 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:20.053672 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.053691 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.054872 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.054876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.056098 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.056110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.056872 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.056880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.057150 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.057794 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:19:20.057817 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:20.059957 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:20.062917 (+ 2960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:20.062927 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:20.062944 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:20.063454 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:20.063844 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:20.063849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:20.063853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:20.064130 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:20.064673 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:20.064681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:20.065115 (+ 434us) client_negotiation.cc:769] Sending connection context
0811 09:19:20.065358 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 09:19:20.065887 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":366,"threads_started":1}
I20260811 09:19:20.068359 16131 heartbeater.cc:347] Connected to a master server at 127.17.16.126:38529
I20260811 09:19:20.068670 16131 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:20.069430 16131 heartbeater.cc:511] Master 127.17.16.126:38529 requested a full tablet report, sending...
I20260811 09:19:20.070817 15360 ts_manager.cc:194] Registered new tserver with Master: d3bded30fc9f40b59f0f220ad5f0c8f7 (127.17.16.67:40047)
I20260811 09:19:20.071597 15360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:53549
I20260811 09:19:20.077039 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:19:20.093420 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.081182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.17.16.126:38529)
0811 09:19:20.081338 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.081343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.081799 (+ 456us) server_negotiation.cc:407] Connection header received
0811 09:19:20.081932 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.081935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.082007 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.082087 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:20.083442 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.084187 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.085439 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.085862 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.085949 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.088374 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:20.088379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:20.088382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:20.088413 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:20.090986 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:20.091753 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:20.091757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:20.091759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:20.091887 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:20.092212 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:20.092216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:20.092218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:20.092525 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:20.092732 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.093048 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.093193 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:19:20.097236 15360 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:19:20.104156 15360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:19:20.129880 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.116369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:40047 (local address 127.0.0.1:60018)
0811 09:19:20.117241 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.117284 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:20.117468 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:20.118967 (+ 1499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:20.118976 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:20.119011 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:20.119461 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.119473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.122833 (+ 3360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.122840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.126676 (+ 3836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.126702 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.128877 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.128884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.129134 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.129191 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 09:19:20.129331 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 09:19:20.129527 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":450,"threads_started":1}
I20260811 09:19:20.129889 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.116631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.17.16.67:40047)
0811 09:19:20.117984 (+ 1353us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.117994 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.118025 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:19:20.118123 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.118131 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.118414 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.118621 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:20.120849 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.122644 (+ 1795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.126911 (+ 4267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.128630 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.128753 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.129026 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.129396 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.129576 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1182,"thread_start_us":1005,"threads_started":1}
I20260811 09:19:20.133929 16016 tablet_service.cc:1467] Processing CreateTablet for tablet 8f92534354954c419840f87a52381be2 (DEFAULT_TABLE table=test-table [id=d4df5559a7da4c8fb7a5a592cdf2a3a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:20.135246 16159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.117722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:45533 (local address 127.0.0.1:48894)
0811 09:19:20.118218 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.118239 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:20.118342 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:20.128045 (+ 9703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:20.128049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:20.128088 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:20.128587 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.128600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.130392 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.130398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.132474 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.132489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.134351 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.134357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.134527 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.134561 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:19:20.134656 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 09:19:20.134798 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":28,"spinlock_wait_cycles":3456,"thread_start_us":250,"threads_started":1}
I20260811 09:19:20.135532 16016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f92534354954c419840f87a52381be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:20.135558 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.118003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48894 (local address 127.17.16.66:45533)
0811 09:19:20.127514 (+ 9511us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.127521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.127545 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:19:20.127636 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.127642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.127814 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.128590 (+ 776us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:20.128836 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.130265 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.132657 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.134109 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.134266 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.134462 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.135069 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.135241 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9358,"thread_start_us":9248,"threads_started":1}
I20260811 09:19:20.135958 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.118286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35695 (local address 127.0.0.1:38686)
0811 09:19:20.127154 (+ 8868us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.127175 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:20.127301 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:20.128040 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:20.128045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:20.128081 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:20.128792 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.128803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.130759 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.130764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.132946 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.132960 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.134795 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.134802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.134973 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.135004 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:19:20.135097 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 09:19:20.135245 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8753,"mutex_wait_us":273,"spinlock_wait_cycles":1408,"thread_start_us":8663,"threads_started":1}
I20260811 09:19:20.136693 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.118827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38686 (local address 127.17.16.65:35695)
0811 09:19:20.126913 (+ 8086us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.126922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.127433 (+ 511us) server_negotiation.cc:407] Connection header received
0811 09:19:20.127554 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.127563 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.127832 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.128171 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:20.129006 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.130619 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.133154 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.134537 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.134732 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.134953 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.136232 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.136394 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7907,"thread_start_us":7746,"threads_started":1}
I20260811 09:19:20.139153 15791 tablet_service.cc:1467] Processing CreateTablet for tablet 8f92534354954c419840f87a52381be2 (DEFAULT_TABLE table=test-table [id=d4df5559a7da4c8fb7a5a592cdf2a3a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:20.140027 15573 tablet_service.cc:1467] Processing CreateTablet for tablet 8f92534354954c419840f87a52381be2 (DEFAULT_TABLE table=test-table [id=d4df5559a7da4c8fb7a5a592cdf2a3a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:20.140621 15791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f92534354954c419840f87a52381be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:20.141367 15573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f92534354954c419840f87a52381be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:20.148310 16165 tablet_bootstrap.cc:492] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: Bootstrap starting.
I20260811 09:19:20.151314 16165 tablet_bootstrap.cc:654] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:20.152284 16165 log.cc:824] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:20.155503 16167 tablet_bootstrap.cc:492] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: Bootstrap starting.
I20260811 09:19:20.156349 16170 tablet_bootstrap.cc:492] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: Bootstrap starting.
I20260811 09:19:20.156380 16165 tablet_bootstrap.cc:492] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: No bootstrap required, opened a new log
I20260811 09:19:20.156667 16165 ts_tablet_manager.cc:1397] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 09:19:20.160014 16170 tablet_bootstrap.cc:654] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:20.160013 16167 tablet_bootstrap.cc:654] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:20.161288 16170 log.cc:824] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:20.161288 16167 log.cc:824] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:20.164062 16170 tablet_bootstrap.cc:492] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: No bootstrap required, opened a new log
I20260811 09:19:20.164306 16170 ts_tablet_manager.cc:1397] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:19:20.164453 16167 tablet_bootstrap.cc:492] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: No bootstrap required, opened a new log
I20260811 09:19:20.164728 16167 ts_tablet_manager.cc:1397] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 09:19:20.164597 16165 raft_consensus.cc:348] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40047 } } peers { permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 } } peers { permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 } }
I20260811 09:19:20.165086 16165 raft_consensus.cc:374] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:20.165190 16165 raft_consensus.cc:729] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3bded30fc9f40b59f0f220ad5f0c8f7, State: Initialized, Role: FOLLOWER
I20260811 09:19:20.165849 16165 consensus_queue.cc:260] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40047 } } peers { permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 } } peers { permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 } }
I20260811 09:19:20.168161 16131 heartbeater.cc:503] Master 127.17.16.126:38529 was elected leader, sending a full tablet report...
I20260811 09:19:20.168939 16165 ts_tablet_manager.cc:1428] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 09:19:20.173082 16170 raft_consensus.cc:348] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40047 } } peers { permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 } } peers { permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 } }
I20260811 09:19:20.173532 16170 raft_consensus.cc:374] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:20.173645 16170 raft_consensus.cc:729] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13950d04616c4f7da198df24972f5266, State: Initialized, Role: FOLLOWER
I20260811 09:19:20.174295 16170 consensus_queue.cc:260] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40047 } } peers { permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 } } peers { permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 } }
I20260811 09:19:20.174419 16167 raft_consensus.cc:348] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40047 } } peers { permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 } } peers { permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 } }
I20260811 09:19:20.174806 16167 raft_consensus.cc:374] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:20.174906 16167 raft_consensus.cc:729] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49096c43331f48da9a44cab454617e72, State: Initialized, Role: FOLLOWER
I20260811 09:19:20.175493 16167 consensus_queue.cc:260] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40047 } } peers { permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 } } peers { permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 } }
I20260811 09:19:20.175666 15906 heartbeater.cc:503] Master 127.17.16.126:38529 was elected leader, sending a full tablet report...
I20260811 09:19:20.176326 16170 ts_tablet_manager.cc:1428] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 09:19:20.177238 16167 ts_tablet_manager.cc:1428] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: Time spent starting tablet: real 0.012s user 0.010s sys 0.004s
I20260811 09:19:20.178141 16175 raft_consensus.cc:482] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:19:20.178459 16175 raft_consensus.cc:504] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40047 } } peers { permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 } } peers { permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 } }
I20260811 09:19:20.180656 16175 leader_election.cc:288] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13950d04616c4f7da198df24972f5266 (127.17.16.66:45533), 49096c43331f48da9a44cab454617e72 (127.17.16.65:35695)
I20260811 09:19:20.186964 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.181695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:45533 (local address 127.17.16.67:59879)
0811 09:19:20.181896 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.181916 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:20.182042 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:20.182628 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:20.182633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:20.182664 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:20.182828 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.182836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.183821 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.183827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.185169 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.185186 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.186271 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.186277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.186448 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.186473 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:19:20.186553 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 09:19:20.186684 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 09:19:20.186961 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.181793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:59879 (local address 127.17.16.66:45533)
0811 09:19:20.181977 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.181983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.182002 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:19:20.182164 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.182170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.182246 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.182321 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:20.182960 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.183686 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.185322 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.186124 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.186268 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.186343 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.186592 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.186726 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:19:20.188016 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.182550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:46039 (local address 127.17.16.65:35695)
0811 09:19:20.182749 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.182755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.183223 (+ 468us) server_negotiation.cc:407] Connection header received
0811 09:19:20.183363 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.183368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.183431 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.183500 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:20.184031 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.184729 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.186373 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.187224 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.187389 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.187454 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.187664 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.187789 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 09:19:20.188038 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.182090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35695 (local address 127.17.16.67:46039)
0811 09:19:20.183131 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.183145 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:20.183261 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:20.183528 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:20.183532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:20.183554 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:20.183889 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.183899 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.184868 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.184873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.186226 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:20.186239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.187387 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.187391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:20.187505 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.187527 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 09:19:20.187609 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 09:19:20.187726 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"thread_start_us":771,"threads_started":1}
I20260811 09:19:20.187966 15824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f92534354954c419840f87a52381be2" candidate_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13950d04616c4f7da198df24972f5266" is_pre_election: true
I20260811 09:19:20.188506 15824 raft_consensus.cc:2454] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3bded30fc9f40b59f0f220ad5f0c8f7 in term 0.
I20260811 09:19:20.188939 15612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f92534354954c419840f87a52381be2" candidate_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49096c43331f48da9a44cab454617e72" is_pre_election: true
I20260811 09:19:20.189246 15940 leader_election.cc:302] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13950d04616c4f7da198df24972f5266, d3bded30fc9f40b59f0f220ad5f0c8f7; no voters:
I20260811 09:19:20.189467 15612 raft_consensus.cc:2454] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3bded30fc9f40b59f0f220ad5f0c8f7 in term 0.
I20260811 09:19:20.189659 16175 raft_consensus.cc:2781] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:19:20.189810 16175 raft_consensus.cc:482] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:19:20.189921 16175 raft_consensus.cc:3037] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:20.192131 16175 raft_consensus.cc:504] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40047 } } peers { permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 } } peers { permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 } }
I20260811 09:19:20.192723 16175 leader_election.cc:288] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [CANDIDATE]: Term 1 election: Requested vote from peers 13950d04616c4f7da198df24972f5266 (127.17.16.66:45533), 49096c43331f48da9a44cab454617e72 (127.17.16.65:35695)
I20260811 09:19:20.193120 15824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f92534354954c419840f87a52381be2" candidate_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13950d04616c4f7da198df24972f5266"
I20260811 09:19:20.193195 15612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f92534354954c419840f87a52381be2" candidate_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49096c43331f48da9a44cab454617e72"
I20260811 09:19:20.193326 15824 raft_consensus.cc:3037] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:20.193385 15612 raft_consensus.cc:3037] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:20.196503 15824 raft_consensus.cc:2454] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3bded30fc9f40b59f0f220ad5f0c8f7 in term 1.
I20260811 09:19:20.196945 15940 leader_election.cc:302] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13950d04616c4f7da198df24972f5266, d3bded30fc9f40b59f0f220ad5f0c8f7; no voters:
I20260811 09:19:20.197185 15612 raft_consensus.cc:2454] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3bded30fc9f40b59f0f220ad5f0c8f7 in term 1.
I20260811 09:19:20.197243 16175 raft_consensus.cc:2781] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:20.198422 16175 raft_consensus.cc:686] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 1 LEADER]: Becoming Leader. State: Replica: d3bded30fc9f40b59f0f220ad5f0c8f7, State: Running, Role: LEADER
I20260811 09:19:20.199337 16175 consensus_queue.cc:237] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40047 } } peers { permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 } } peers { permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 } }
I20260811 09:19:20.207021 15355 catalog_manager.cc:5495] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3bded30fc9f40b59f0f220ad5f0c8f7 (127.17.16.67). New cstate: current_term: 1 leader_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3bded30fc9f40b59f0f220ad5f0c8f7" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 } health_report { overall_health: UNKNOWN } } }
W20260811 09:19:20.210829 15679 tablet.cc:2369] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:19:20.229578 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.224685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60032 (local address 127.17.16.67:40047)
0811 09:19:20.224880 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.224885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.224924 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:19:20.225084 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.225087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.225180 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.225308 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:19:20.225638 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.226326 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.227600 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.227969 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.228051 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.228476 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:19:20.229051 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:19:20.229180 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.229263 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.229303 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:19:20.237514 15612 raft_consensus.cc:1261] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72 [term 1 FOLLOWER]: Refusing update from remote peer d3bded30fc9f40b59f0f220ad5f0c8f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:20.237541 15824 raft_consensus.cc:1261] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266 [term 1 FOLLOWER]: Refusing update from remote peer d3bded30fc9f40b59f0f220ad5f0c8f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:20.238442 16180 consensus_queue.cc:1035] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13950d04616c4f7da198df24972f5266" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:20.238809 16175 consensus_queue.cc:1035] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49096c43331f48da9a44cab454617e72" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 35695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:20.263688 15355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:20.263965 15355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:20.267462 15355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3c445dae6d14fe18069dde4d7239a51: Sending DeleteTablet for 3 replicas of tablet 8f92534354954c419840f87a52381be2
I20260811 09:19:20.268527 16016 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f92534354954c419840f87a52381be2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:20 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:60018
I20260811 09:19:20.268801 15791 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f92534354954c419840f87a52381be2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:20 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:48894
I20260811 09:19:20.270006 15573 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f92534354954c419840f87a52381be2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:20 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:38686
I20260811 09:19:20.270128 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 15470
I20260811 09:19:20.280208 16203 tablet_replica.cc:330] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: stopping tablet replica
I20260811 09:19:20.280884 16203 raft_consensus.cc:2229] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:20.281653 16203 raft_consensus.cc:2258] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:20.283870 16202 tablet_replica.cc:330] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: stopping tablet replica
I20260811 09:19:20.283895 16203 ts_tablet_manager.cc:1905] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:20.284307 16202 raft_consensus.cc:2229] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:19:20.285439 16202 raft_consensus.cc:2258] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:20.290534 16204 tablet_replica.cc:330] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: stopping tablet replica
I20260811 09:19:20.290970 16204 raft_consensus.cc:2229] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:20.291515 16204 raft_consensus.cc:2258] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:20.292164 16202 ts_tablet_manager.cc:1905] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:20.293758 16204 ts_tablet_manager.cc:1905] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:19:20 dist-test-slave-ncp9 krb5kdc[15286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:19:20.294814 16203 ts_tablet_manager.cc:1918] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:20.295053 16203 log.cc:1196] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/8f92534354954c419840f87a52381be2
I20260811 09:19:20.295696 16203 ts_tablet_manager.cc:1939] T 8f92534354954c419840f87a52381be2 P 13950d04616c4f7da198df24972f5266: Deleting consensus metadata
I20260811 09:19:20.299314 15330 catalog_manager.cc:4841] TS 13950d04616c4f7da198df24972f5266 (127.17.16.66:45533): tablet 8f92534354954c419840f87a52381be2 (table test-table [id=d4df5559a7da4c8fb7a5a592cdf2a3a7]) successfully deleted
I20260811 09:19:20.299511 16202 ts_tablet_manager.cc:1918] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:20.299676 16202 log.cc:1196] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/8f92534354954c419840f87a52381be2
I20260811 09:19:20.300244 16202 ts_tablet_manager.cc:1939] T 8f92534354954c419840f87a52381be2 P d3bded30fc9f40b59f0f220ad5f0c8f7: Deleting consensus metadata
I20260811 09:19:20.301950 15330 catalog_manager.cc:4841] TS d3bded30fc9f40b59f0f220ad5f0c8f7 (127.17.16.67:40047): tablet 8f92534354954c419840f87a52381be2 (table test-table [id=d4df5559a7da4c8fb7a5a592cdf2a3a7]) successfully deleted
I20260811 09:19:20.303846 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.271568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.17.16.65:35695)
0811 09:19:20.271754 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.271760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.283048 (+ 11288us) server_negotiation.cc:407] Connection header received
0811 09:19:20.283575 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.283583 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.283681 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.283760 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:20.285844 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.287154 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.289305 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.290076 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.290193 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.296084 (+ 5891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:20.296099 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:20.296115 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:20.296168 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:20.299963 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:20.301110 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:20.301120 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:20.301123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:20.301451 (+ 328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:20.301968 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:20.301974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:20.301977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:20.302459 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:20.302695 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.303214 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.303462 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:19:20.304507 15672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:20.418111 16204 ts_tablet_manager.cc:1918] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:20.418406 16204 log.cc:1196] T 8f92534354954c419840f87a52381be2 P 49096c43331f48da9a44cab454617e72: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/8f92534354954c419840f87a52381be2
I20260811 09:19:20.418784 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 15470
W20260811 09:19:20.444020 15330 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.16.65:35695 (error 108)
I20260811 09:19:20.444200 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 15687
I20260811 09:19:20.446152 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.445662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35695 (local address 127.0.0.1:38700)
0811 09:19:20.445853 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.445974 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:35695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 09:19:20.446561 15330 catalog_manager.cc:4584] TS 49096c43331f48da9a44cab454617e72 (127.17.16.65:35695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f92534354954c419840f87a52381be2: Network error: Client connection negotiation failed: client connection to 127.17.16.65:35695: connect: Connection refused (error 111)
Aug 11 09:19:20 dist-test-slave-ncp9 krb5kdc[15286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:19:20.461980 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.445209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.17.16.66:45533)
0811 09:19:20.445436 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.445442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.445894 (+ 452us) server_negotiation.cc:407] Connection header received
0811 09:19:20.446082 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.446087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.446187 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.446295 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:20.447766 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.448968 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.450275 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.450892 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.451043 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.455172 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:20.455194 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:20.455215 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:20.455261 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:20.459094 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:20.459824 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:20.459834 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:20.459838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:20.459992 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:20.460331 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:20.460336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:20.460340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:20.460836 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:20.461213 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.461378 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.461616 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:19:20.462860 15895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:20.500717 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.500123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35695 (local address 127.0.0.1:38704)
0811 09:19:20.500397 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.500523 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:35695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 09:19:20.502142 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.501592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35695 (local address 127.0.0.1:38716)
0811 09:19:20.501850 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.501972 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:35695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 09:19:20.551211 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 15687
I20260811 09:19:20.564700 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.563988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35695 (local address 127.0.0.1:38730)
0811 09:19:20.564349 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.564477 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:35695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 09:19:20.566164 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.565652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35695 (local address 127.0.0.1:38744)
0811 09:19:20.565930 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.566021 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:35695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260811 09:19:20.590112 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 15917
Aug 11 09:19:20 dist-test-slave-ncp9 krb5kdc[15286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:19:20.607700 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.591010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60046 (local address 127.17.16.67:40047)
0811 09:19:20.591208 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:20.591215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:20.591238 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:19:20.591320 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:20.591326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:20.591409 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:20.591516 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:20.592816 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.594081 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.595341 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:20.596183 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:20.596327 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:20.600513 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:20.600531 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:20.600552 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:20.600603 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:20.604734 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:20.605459 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:20.605469 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:20.605472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:20.605667 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:20.606052 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:20.606057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:20.606061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:20.606534 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:20.606876 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:20.607047 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 09:19:20.607296 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:19:20.608529 16123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:20.647414 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.646855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35695 (local address 127.0.0.1:38760)
0811 09:19:20.647125 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.647235 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:35695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 09:19:20.648613 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.648195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35695 (local address 127.0.0.1:38772)
0811 09:19:20.648408 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.648493 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:35695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 09:19:20.732934 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 15917
I20260811 09:19:20.759977 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 15306
I20260811 09:19:20.761175 15412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:20.871604 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:20.871004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:35695 (local address 127.0.0.1:38778)
0811 09:19:20.871293 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:20.871430 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:35695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260811 09:19:20.871672 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 15306
2026-08-11T09:19:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4607 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T09:19:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:20Z Disabled control of system clock
I20260811 09:19:20.970700 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:38999
--webserver_interface=127.17.16.126
--webserver_port=0
--builtin_ntp_servers=127.17.16.84:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:38999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:19:21.313475 16222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:21.313766 16222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:21.313848 16222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:21.321065 16222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:21.321180 16222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:21.321225 16222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:21.321282 16222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:21.321326 16222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:21.330652 16222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:38999
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:38999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:21.331976 16222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:21.333841 16222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:21.345624 16231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:21.345650 16230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:21.345665 16235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:21.346472 16222 server_base.cc:1034] running on GCE node
I20260811 09:19:21.347306 16222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:21.348820 16222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:21.350054 16222 hybrid_clock.cc:648] HybridClock initialized: now 1786439961349998 us; error 49 us; skew 500 ppm
I20260811 09:19:21.350427 16222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:21.352677 16222 webserver.cc:460] Webserver started at http://127.17.16.126:42591/ using document root <none> and password file <none>
I20260811 09:19:21.353404 16222 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:21.353508 16222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:21.353821 16222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:21.356786 16222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "928d11b894aa4b9e916a0a13da804f94"
format_stamp: "Formatted at 2026-08-11 09:19:21 on dist-test-slave-ncp9"
server_key: "733ed5a5e2e8cf67cb304f472483203c"
server_key_iv: "1c666d35f8845b8da2e5fdd7b5c26b34"
server_key_version: "encryptionkey@0"
I20260811 09:19:21.357659 16222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "928d11b894aa4b9e916a0a13da804f94"
format_stamp: "Formatted at 2026-08-11 09:19:21 on dist-test-slave-ncp9"
server_key: "733ed5a5e2e8cf67cb304f472483203c"
server_key_iv: "1c666d35f8845b8da2e5fdd7b5c26b34"
server_key_version: "encryptionkey@0"
I20260811 09:19:21.362886 16222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:19:21.366333 16244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:21.367654 16222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 09:19:21.367831 16222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "928d11b894aa4b9e916a0a13da804f94"
format_stamp: "Formatted at 2026-08-11 09:19:21 on dist-test-slave-ncp9"
server_key: "733ed5a5e2e8cf67cb304f472483203c"
server_key_iv: "1c666d35f8845b8da2e5fdd7b5c26b34"
server_key_version: "encryptionkey@0"
I20260811 09:19:21.368065 16222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:21.406705 16222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:21.407599 16222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:21.407912 16222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:21.425529 16222 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:38999
I20260811 09:19:21.425582 16327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:38999 every 8 connection(s)
I20260811 09:19:21.426888 16222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:21.430739 16330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:21.431861 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 16222
I20260811 09:19:21.432018 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:21.432464 17473 external_mini_cluster.cc:1428] Setting key 5914ff8fc8c2e54de11a656d0ea90a16
I20260811 09:19:21.445122 16330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94: Bootstrap starting.
I20260811 09:19:21.447469 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:21.434358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50334 (local address 127.17.16.126:38999)
0811 09:19:21.435366 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:21.435378 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:21.435424 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:19:21.437081 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:21.437106 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:21.437572 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:21.437911 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:21.438547 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:21.439888 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:21.441853 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:21.442584 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:21.442794 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:21.444251 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:21.444262 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:21.444282 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:21.445014 (+ 732us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:21.445137 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:21.445165 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:21.445273 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:21.445694 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 09:19:21.446148 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":359,"threads_started":1}
I20260811 09:19:21.449507 16330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:21.450841 16330 log.cc:824] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:21.453765 16330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94: No bootstrap required, opened a new log
I20260811 09:19:21.461031 16330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928d11b894aa4b9e916a0a13da804f94" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38999 } }
I20260811 09:19:21.461447 16330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:21.461529 16330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928d11b894aa4b9e916a0a13da804f94, State: Initialized, Role: FOLLOWER
I20260811 09:19:21.462044 16330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928d11b894aa4b9e916a0a13da804f94" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38999 } }
I20260811 09:19:21.462273 16330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:21.462378 16330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:21.462502 16330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:21.464597 16330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928d11b894aa4b9e916a0a13da804f94" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38999 } }
I20260811 09:19:21.465109 16330 leader_election.cc:302] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 928d11b894aa4b9e916a0a13da804f94; no voters:
I20260811 09:19:21.465653 16330 leader_election.cc:288] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:21.465763 16341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:21.466427 16341 raft_consensus.cc:686] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [term 1 LEADER]: Becoming Leader. State: Replica: 928d11b894aa4b9e916a0a13da804f94, State: Running, Role: LEADER
I20260811 09:19:21.466993 16341 consensus_queue.cc:237] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928d11b894aa4b9e916a0a13da804f94" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38999 } }
I20260811 09:19:21.467871 16330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:21.469897 16344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 928d11b894aa4b9e916a0a13da804f94. Latest consensus state: current_term: 1 leader_uuid: "928d11b894aa4b9e916a0a13da804f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928d11b894aa4b9e916a0a13da804f94" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38999 } } }
I20260811 09:19:21.469866 16343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "928d11b894aa4b9e916a0a13da804f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928d11b894aa4b9e916a0a13da804f94" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 38999 } } }
I20260811 09:19:21.470309 16343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:21.470306 16344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:21.477797 16350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:21.482254 16350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:21.490883 16350 catalog_manager.cc:1347] Generated new cluster ID: 3b17a6070e064356b821250677c30e76
I20260811 09:19:21.491007 16350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:21.506074 16350 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:21.507951 16350 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:21.522908 16350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94: Generated new TSK 0
I20260811 09:19:21.524125 16350 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:21.537345 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38999
--builtin_ntp_servers=127.17.16.84:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:19:21.881922 16365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:21.882233 16365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:21.882330 16365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:21.889531 16365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:21.889638 16365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:21.889750 16365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:19:21.899108 16365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:21.900357 16365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:21.902138 16365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:21.914909 16377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:21.915102 16381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:21.915141 16375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:21.915745 16365 server_base.cc:1034] running on GCE node
I20260811 09:19:21.916349 16365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:21.917353 16365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:21.918565 16365 hybrid_clock.cc:648] HybridClock initialized: now 1786439961918511 us; error 36 us; skew 500 ppm
I20260811 09:19:21.918870 16365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:21.921046 16365 webserver.cc:460] Webserver started at http://127.17.16.65:37981/ using document root <none> and password file <none>
I20260811 09:19:21.921732 16365 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:21.921838 16365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:21.922163 16365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:21.925777 16365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "958e2eb5924a435287416dd1fefd31ef"
format_stamp: "Formatted at 2026-08-11 09:19:21 on dist-test-slave-ncp9"
server_key: "ae2b00ea6b719aa6525b3990632ed440"
server_key_iv: "48939f072608b8ebe85be1f51c2af26a"
server_key_version: "encryptionkey@0"
I20260811 09:19:21.926745 16365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "958e2eb5924a435287416dd1fefd31ef"
format_stamp: "Formatted at 2026-08-11 09:19:21 on dist-test-slave-ncp9"
server_key: "ae2b00ea6b719aa6525b3990632ed440"
server_key_iv: "48939f072608b8ebe85be1f51c2af26a"
server_key_version: "encryptionkey@0"
I20260811 09:19:21.931999 16365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:19:21.935554 16396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:21.936854 16365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:19:21.937042 16365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "958e2eb5924a435287416dd1fefd31ef"
format_stamp: "Formatted at 2026-08-11 09:19:21 on dist-test-slave-ncp9"
server_key: "ae2b00ea6b719aa6525b3990632ed440"
server_key_iv: "48939f072608b8ebe85be1f51c2af26a"
server_key_version: "encryptionkey@0"
I20260811 09:19:21.937232 16365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:21.953357 16365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:21.954123 16365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:21.954382 16365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:21.955271 16365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:21.956672 16365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:21.956759 16365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:21.956840 16365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:21.956921 16365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:21.987335 16365 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:36239
I20260811 09:19:21.987385 16585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:36239 every 8 connection(s)
I20260811 09:19:21.988607 16365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:19:21.989801 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 16365
I20260811 09:19:21.989965 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:19:21.990435 17473 external_mini_cluster.cc:1428] Setting key 84012ac0415bb08c787113ba4904fe6a
I20260811 09:19:21.993850 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38999
--builtin_ntp_servers=127.17.16.84:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:19:22.001264 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:21.992365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:59075 (local address 127.17.16.126:38999)
0811 09:19:21.992817 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:21.992824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:21.993136 (+ 312us) server_negotiation.cc:407] Connection header received
0811 09:19:21.994924 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:21.994930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:21.995007 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:21.995132 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:21.996133 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:21.997106 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:21.998608 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:21.999128 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:21.999243 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:22.000464 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:22.000470 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:22.000475 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:22.000496 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:22.000542 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:22.000556 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:22.000633 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:22.000817 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 09:19:22.000934 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":630,"threads_started":1}
I20260811 09:19:22.002220 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:21.991812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:38999 (local address 127.17.16.65:59075)
0811 09:19:21.992961 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:21.993020 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:21.994658 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:21.995326 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:21.995334 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:21.995418 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:21.995971 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:21.995988 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:21.997232 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:21.997236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:21.998464 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:21.998476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:21.999304 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:21.999308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:21.999509 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:22.000162 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:19:22.000165 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:22.000203 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:19:22.000279 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:22.000641 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:22.000649 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:22.000654 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 09:19:22.000784 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 09:19:22.001109 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":335,"threads_started":1}
I20260811 09:19:22.003681 16586 heartbeater.cc:347] Connected to a master server at 127.17.16.126:38999
I20260811 09:19:22.003948 16586 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:22.004693 16586 heartbeater.cc:511] Master 127.17.16.126:38999 requested a full tablet report, sending...
I20260811 09:19:22.006775 16272 ts_manager.cc:194] Registered new tserver with Master: 958e2eb5924a435287416dd1fefd31ef (127.17.16.65:36239)
I20260811 09:19:22.008241 16272 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.65:59075
W20260811 09:19:22.331132 16590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:22.331418 16590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:22.331589 16590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:22.338757 16590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:22.338867 16590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:22.339022 16590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:19:22.348201 16590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:22.349524 16590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:22.351315 16590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:22.363958 16597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:22.363919 16602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:22.364010 16599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:22.365177 16590 server_base.cc:1034] running on GCE node
I20260811 09:19:22.365808 16590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:22.366730 16590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:22.367921 16590 hybrid_clock.cc:648] HybridClock initialized: now 1786439962367873 us; error 39 us; skew 500 ppm
I20260811 09:19:22.368234 16590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:22.370472 16590 webserver.cc:460] Webserver started at http://127.17.16.66:46401/ using document root <none> and password file <none>
I20260811 09:19:22.371155 16590 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:22.371249 16590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:22.371577 16590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:22.374454 16590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "c448b5572a5a46148f2adb22e6302934"
format_stamp: "Formatted at 2026-08-11 09:19:22 on dist-test-slave-ncp9"
server_key: "aabd5d97432878aa7787684e60b66c6f"
server_key_iv: "8d8708779cbe91d2201d1a4f5049e64a"
server_key_version: "encryptionkey@0"
I20260811 09:19:22.375205 16590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "c448b5572a5a46148f2adb22e6302934"
format_stamp: "Formatted at 2026-08-11 09:19:22 on dist-test-slave-ncp9"
server_key: "aabd5d97432878aa7787684e60b66c6f"
server_key_iv: "8d8708779cbe91d2201d1a4f5049e64a"
server_key_version: "encryptionkey@0"
I20260811 09:19:22.380183 16590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:19:22.383483 16613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:22.384843 16590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:19:22.385037 16590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "c448b5572a5a46148f2adb22e6302934"
format_stamp: "Formatted at 2026-08-11 09:19:22 on dist-test-slave-ncp9"
server_key: "aabd5d97432878aa7787684e60b66c6f"
server_key_iv: "8d8708779cbe91d2201d1a4f5049e64a"
server_key_version: "encryptionkey@0"
I20260811 09:19:22.385233 16590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:22.404363 16590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:22.405189 16590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:22.405458 16590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:22.406457 16590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:22.407893 16590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:22.407981 16590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:22.408061 16590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:22.408113 16590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:22.439793 16590 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:35513
I20260811 09:19:22.439843 16799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:35513 every 8 connection(s)
I20260811 09:19:22.441596 16590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:19:22.444028 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 16590
I20260811 09:19:22.444204 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:19:22.444706 17473 external_mini_cluster.cc:1428] Setting key 809777bd690252805dad42644a9c4645
I20260811 09:19:22.447786 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38999
--builtin_ntp_servers=127.17.16.84:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:19:22.455094 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.444167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:45021 (local address 127.17.16.126:38999)
0811 09:19:22.444368 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:22.444374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:22.445927 (+ 1553us) server_negotiation.cc:407] Connection header received
0811 09:19:22.448214 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:22.448218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:22.448282 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:22.448359 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:22.449655 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.450344 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.452381 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.452791 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.452885 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:22.454388 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:22.454393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:22.454396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:22.454414 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:22.454446 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:22.454458 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:22.454524 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:22.454769 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 09:19:22.454852 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:19:22.456416 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.444705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:38999 (local address 127.17.16.66:45021)
0811 09:19:22.445754 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:22.445812 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:22.447942 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:22.448579 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:22.448590 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:22.448701 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:22.449496 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:22.449520 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.450489 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.450494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:22.452231 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:22.452246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.453032 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.453039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:22.453313 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:22.454026 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:19:22.454030 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:22.454078 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:19:22.454205 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:22.454543 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:22.454552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:22.454559 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 09:19:22.454726 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 09:19:22.455105 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":342,"threads_started":1}
I20260811 09:19:22.457815 16800 heartbeater.cc:347] Connected to a master server at 127.17.16.126:38999
I20260811 09:19:22.458101 16800 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:22.458835 16800 heartbeater.cc:511] Master 127.17.16.126:38999 requested a full tablet report, sending...
I20260811 09:19:22.460210 16272 ts_manager.cc:194] Registered new tserver with Master: c448b5572a5a46148f2adb22e6302934 (127.17.16.66:35513)
I20260811 09:19:22.461246 16272 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.66:45021
W20260811 09:19:22.791115 16803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:22.791431 16803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:22.791577 16803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:22.798833 16803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:22.798945 16803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:22.799096 16803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:19:22.808355 16803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:38999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:22.809721 16803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:22.811522 16803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:22.824200 16817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:22.824281 16813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:22.824250 16814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:22.825183 16803 server_base.cc:1034] running on GCE node
I20260811 09:19:22.825788 16803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:22.826777 16803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:22.827986 16803 hybrid_clock.cc:648] HybridClock initialized: now 1786439962827927 us; error 51 us; skew 500 ppm
I20260811 09:19:22.828312 16803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:22.830556 16803 webserver.cc:460] Webserver started at http://127.17.16.67:39797/ using document root <none> and password file <none>
I20260811 09:19:22.831241 16803 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:22.831344 16803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:22.831665 16803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:22.834580 16803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "2510ddaf9fda4f59a9d639c5394bb318"
format_stamp: "Formatted at 2026-08-11 09:19:22 on dist-test-slave-ncp9"
server_key: "5eed55ba7f64dc956d384a142f804f62"
server_key_iv: "614a577123347685db2e266290739234"
server_key_version: "encryptionkey@0"
I20260811 09:19:22.835357 16803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "2510ddaf9fda4f59a9d639c5394bb318"
format_stamp: "Formatted at 2026-08-11 09:19:22 on dist-test-slave-ncp9"
server_key: "5eed55ba7f64dc956d384a142f804f62"
server_key_iv: "614a577123347685db2e266290739234"
server_key_version: "encryptionkey@0"
I20260811 09:19:22.840438 16803 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:19:22.843950 16829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:22.845253 16803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:19:22.845399 16803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "2510ddaf9fda4f59a9d639c5394bb318"
format_stamp: "Formatted at 2026-08-11 09:19:22 on dist-test-slave-ncp9"
server_key: "5eed55ba7f64dc956d384a142f804f62"
server_key_iv: "614a577123347685db2e266290739234"
server_key_version: "encryptionkey@0"
I20260811 09:19:22.845590 16803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:22.878603 16803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:22.879592 16803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:22.879894 16803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:22.880959 16803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:22.882457 16803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:22.882561 16803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:22.882668 16803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:22.882738 16803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:22.913085 16803 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:43709
I20260811 09:19:22.913117 17016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:43709 every 8 connection(s)
I20260811 09:19:22.914346 16803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:19:22.919456 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 16803
I20260811 09:19:22.919606 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:19:22.920081 17473 external_mini_cluster.cc:1428] Setting key 74c77f90554ef6bf4712603e05aa6548
I20260811 09:19:22.926527 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.916824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:45923 (local address 127.17.16.126:38999)
0811 09:19:22.917022 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:22.917026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:22.918383 (+ 1357us) server_negotiation.cc:407] Connection header received
0811 09:19:22.920215 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:22.920221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:22.920301 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:22.920391 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:22.921585 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.922440 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.923971 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.924379 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.924488 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:22.925739 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:22.925744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:22.925748 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:22.925765 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:22.925797 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:22.925808 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:22.925881 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:22.926104 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 09:19:22.926212 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:19:22.927764 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.917360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:38999 (local address 127.17.16.67:45923)
0811 09:19:22.918227 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:22.918264 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:22.919943 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:22.920655 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:22.920663 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:22.920759 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:22.921422 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:22.921440 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.922588 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.922592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:22.923829 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:22.923842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.924592 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.924600 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:22.924818 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:22.925441 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:19:22.925444 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:22.925482 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:19:22.925558 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:22.925896 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:22.925907 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:22.925914 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 09:19:22.926058 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 09:19:22.926462 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":271,"threads_started":1}
I20260811 09:19:22.929195 17017 heartbeater.cc:347] Connected to a master server at 127.17.16.126:38999
I20260811 09:19:22.929476 17017 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:22.930202 17017 heartbeater.cc:511] Master 127.17.16.126:38999 requested a full tablet report, sending...
I20260811 09:19:22.931500 16272 ts_manager.cc:194] Registered new tserver with Master: 2510ddaf9fda4f59a9d639c5394bb318 (127.17.16.67:43709)
I20260811 09:19:22.932286 16272 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.67:45923
I20260811 09:19:22.934999 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:19:22.944780 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.939065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50344 (local address 127.17.16.126:38999)
0811 09:19:22.939268 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:22.939275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:22.939608 (+ 333us) server_negotiation.cc:407] Connection header received
0811 09:19:22.939752 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:22.939755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:22.939825 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:22.939899 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:22.940345 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.941100 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.942534 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.943023 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.943131 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:22.944166 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:22.944171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:22.944174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:22.944190 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:22.944218 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:22.944228 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:22.944332 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:22.944453 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 09:19:22.944497 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:19:22.948443 16272 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:19:22.955205 16272 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:19:23.000711 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.966961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:43709 (local address 127.0.0.1:41392)
0811 09:19:22.991384 (+ 24423us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:22.991425 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:22.991607 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:22.993614 (+ 2007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:22.993622 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:22.993660 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:22.994058 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:22.994073 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.996065 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.996071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:22.997786 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:22.997801 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.999780 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.999786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.000010 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.000067 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 09:19:23.000197 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 09:19:23.000380 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24132,"thread_start_us":26042,"threads_started":1}
I20260811 09:19:23.001140 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.967026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41392 (local address 127.17.16.67:43709)
0811 09:19:22.992871 (+ 25845us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:22.992907 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:22.992949 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:19:22.993056 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:22.993063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:22.993296 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:22.994273 (+ 977us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:22.994579 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.995941 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.998036 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.999570 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.999728 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.000035 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.000267 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.000463 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25666,"thread_start_us":25539,"threads_started":1}
I20260811 09:19:23.002041 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.967017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.0.0.1:34950)
0811 09:19:22.993269 (+ 26252us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:22.993291 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:22.993422 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:22.994602 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:22.994606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:22.994628 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:22.995077 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:22.995092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.997330 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.997336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:22.999397 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:22.999409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.001295 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.001302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.001462 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.001486 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:19:23.001565 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 09:19:23.001692 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26130,"mutex_wait_us":56,"thread_start_us":27300,"threads_started":1}
I20260811 09:19:23.002141 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.967743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.17.16.65:36239)
0811 09:19:22.993991 (+ 26248us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:22.994002 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:22.994037 (+ 35us) server_negotiation.cc:407] Connection header received
0811 09:19:22.994148 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:22.994157 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:22.994363 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:22.995301 (+ 938us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:22.995558 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.997161 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.999579 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.000998 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.001131 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.001397 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.001632 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.001787 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26041,"thread_start_us":25896,"threads_started":1}
I20260811 09:19:23.004163 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.967494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35513 (local address 127.0.0.1:38826)
0811 09:19:22.994598 (+ 27104us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:22.994616 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:22.994746 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:22.995343 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:22.995348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:22.995374 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:22.995739 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:22.995750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.997574 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:22.997580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:22.999823 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:22.999839 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.003501 (+ 3662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.003505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.003623 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.003643 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 09:19:23.003717 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 09:19:23.003838 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26971,"mutex_wait_us":2,"thread_start_us":24352,"threads_started":1}
I20260811 09:19:23.004287 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:22.968107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38826 (local address 127.17.16.66:35513)
0811 09:19:22.990821 (+ 22714us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:22.990830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:22.994741 (+ 3911us) server_negotiation.cc:407] Connection header received
0811 09:19:22.994894 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:22.994903 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:22.995132 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:22.995331 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:22.995925 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:22.997420 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.000089 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.003138 (+ 3049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.003218 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.003367 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.003790 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.003956 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22236,"thread_start_us":22375,"threads_started":1}
I20260811 09:19:23.005316 16907 tablet_service.cc:1467] Processing CreateTablet for tablet 0d91636701304e139ed73178179a1a2b (DEFAULT_TABLE table=test-table [id=a85d3c5002214785aac7a78898cae5af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:23.005717 16472 tablet_service.cc:1467] Processing CreateTablet for tablet 0d91636701304e139ed73178179a1a2b (DEFAULT_TABLE table=test-table [id=a85d3c5002214785aac7a78898cae5af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:23.007014 16907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d91636701304e139ed73178179a1a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:23.007248 16472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d91636701304e139ed73178179a1a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:23.007292 16702 tablet_service.cc:1467] Processing CreateTablet for tablet 0d91636701304e139ed73178179a1a2b (DEFAULT_TABLE table=test-table [id=a85d3c5002214785aac7a78898cae5af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:23.008347 16702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d91636701304e139ed73178179a1a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:23.010988 16586 heartbeater.cc:503] Master 127.17.16.126:38999 was elected leader, sending a full tablet report...
I20260811 09:19:23.021037 17042 tablet_bootstrap.cc:492] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: Bootstrap starting.
I20260811 09:19:23.024410 17042 tablet_bootstrap.cc:654] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:23.025364 17045 tablet_bootstrap.cc:492] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: Bootstrap starting.
I20260811 09:19:23.025722 17042 log.cc:824] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:23.027336 17046 tablet_bootstrap.cc:492] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef: Bootstrap starting.
I20260811 09:19:23.029173 17042 tablet_bootstrap.cc:492] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: No bootstrap required, opened a new log
I20260811 09:19:23.029245 17045 tablet_bootstrap.cc:654] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:23.029456 17042 ts_tablet_manager.cc:1397] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 09:19:23.030537 17045 log.cc:824] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:23.030889 17046 tablet_bootstrap.cc:654] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:23.032135 17046 log.cc:824] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:23.035104 17046 tablet_bootstrap.cc:492] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef: No bootstrap required, opened a new log
I20260811 09:19:23.035192 17045 tablet_bootstrap.cc:492] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: No bootstrap required, opened a new log
I20260811 09:19:23.035356 17046 ts_tablet_manager.cc:1397] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:19:23.035480 17045 ts_tablet_manager.cc:1397] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260811 09:19:23.039230 17042 raft_consensus.cc:348] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.039731 17042 raft_consensus.cc:374] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:23.039853 17042 raft_consensus.cc:729] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c448b5572a5a46148f2adb22e6302934, State: Initialized, Role: FOLLOWER
I20260811 09:19:23.040530 17042 consensus_queue.cc:260] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.042853 17046 raft_consensus.cc:348] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.043185 17046 raft_consensus.cc:374] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:23.043280 17046 raft_consensus.cc:729] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 958e2eb5924a435287416dd1fefd31ef, State: Initialized, Role: FOLLOWER
I20260811 09:19:23.043798 17046 consensus_queue.cc:260] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.045550 17046 ts_tablet_manager.cc:1428] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260811 09:19:23.046514 16800 heartbeater.cc:503] Master 127.17.16.126:38999 was elected leader, sending a full tablet report...
I20260811 09:19:23.046991 17045 raft_consensus.cc:348] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.047344 17042 ts_tablet_manager.cc:1428] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
I20260811 09:19:23.047585 17045 raft_consensus.cc:374] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:23.047690 17045 raft_consensus.cc:729] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2510ddaf9fda4f59a9d639c5394bb318, State: Initialized, Role: FOLLOWER
I20260811 09:19:23.048275 17045 consensus_queue.cc:260] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.049469 17017 heartbeater.cc:503] Master 127.17.16.126:38999 was elected leader, sending a full tablet report...
I20260811 09:19:23.050235 17045 ts_tablet_manager.cc:1428] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: Time spent starting tablet: real 0.015s user 0.007s sys 0.009s
W20260811 09:19:23.165927 17018 tablet.cc:2369] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:19:23.193665 16801 tablet.cc:2369] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:19:23.196267 17052 raft_consensus.cc:482] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:19:23.196611 17052 raft_consensus.cc:504] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.197415 17055 raft_consensus.cc:482] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:19:23.197737 17055 raft_consensus.cc:504] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.198876 17052 leader_election.cc:288] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2510ddaf9fda4f59a9d639c5394bb318 (127.17.16.67:43709), 958e2eb5924a435287416dd1fefd31ef (127.17.16.65:36239)
I20260811 09:19:23.205935 17055 leader_election.cc:288] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 958e2eb5924a435287416dd1fefd31ef (127.17.16.65:36239), c448b5572a5a46148f2adb22e6302934 (127.17.16.66:35513)
I20260811 09:19:23.218917 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.199418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:55815 (local address 127.17.16.65:36239)
0811 09:19:23.199645 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:23.199651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:23.211410 (+ 11759us) server_negotiation.cc:407] Connection header received
0811 09:19:23.212148 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:23.212154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:23.212289 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:23.212350 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:23.213176 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.214337 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.217045 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.217911 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.218056 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.218127 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.218508 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.218645 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:19:23.219267 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.200050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:55635 (local address 127.17.16.67:43709)
0811 09:19:23.200216 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:23.200222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:23.211835 (+ 11613us) server_negotiation.cc:407] Connection header received
0811 09:19:23.212039 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:23.212044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:23.212125 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:23.212216 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:23.213137 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.214351 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.217040 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.218185 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.218308 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.218374 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.218875 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.219017 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":58}
I20260811 09:19:23.219763 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.211238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:53651 (local address 127.17.16.66:35513)
0811 09:19:23.211428 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:23.211433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:23.212535 (+ 1102us) server_negotiation.cc:407] Connection header received
0811 09:19:23.212802 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:23.212807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:23.212922 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:23.213006 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:23.213895 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.214929 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.217348 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.218496 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.218622 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.218685 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.219339 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.219463 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 09:19:23.220090 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.199690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.17.16.67:54769)
0811 09:19:23.199861 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.199888 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:23.200042 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:23.212504 (+ 12462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:23.212509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:23.212538 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:23.212785 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:23.212797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.214704 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.214710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.216761 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:23.216776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.219416 (+ 2640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.219422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.219590 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.219623 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:19:23.219697 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 09:19:23.219818 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"spinlock_wait_cycles":17280}
I20260811 09:19:23.220163 17057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.199108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.17.16.66:55815)
0811 09:19:23.211274 (+ 12166us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.211298 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:23.211991 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:23.212548 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:23.212552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:23.212587 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:23.213024 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:23.213034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.214516 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.214520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.216770 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:23.216789 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.218110 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.218116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.218308 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.218349 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 09:19:23.219543 (+ 1194us) client_negotiation.cc:240] Negotiation successful
0811 09:19:23.219692 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12001,"mutex_wait_us":8,"spinlock_wait_cycles":32640,"thread_start_us":12320,"threads_started":1}
I20260811 09:19:23.220448 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.200785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35513 (local address 127.17.16.67:53651)
0811 09:19:23.212443 (+ 11658us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.212459 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:23.212635 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:23.213223 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:23.213227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:23.213256 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:23.213700 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:23.213714 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.215123 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.215128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.217159 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:23.217178 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.218783 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.218789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.218948 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.218980 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:19:23.219991 (+ 1011us) client_negotiation.cc:240] Negotiation successful
0811 09:19:23.220093 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11530,"mutex_wait_us":1,"thread_start_us":13733,"threads_started":1}
I20260811 09:19:23.220505 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.199226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:43709 (local address 127.17.16.66:55635)
0811 09:19:23.211731 (+ 12505us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.211748 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:23.211901 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:23.212387 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:23.212392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:23.212422 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:23.212922 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:23.212939 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.214511 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.214516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.216800 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:23.216810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.218541 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.218548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:23.218704 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.218731 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 09:19:23.219491 (+ 760us) client_negotiation.cc:240] Negotiation successful
0811 09:19:23.219624 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12401,"mutex_wait_us":203,"thread_start_us":11672,"threads_started":1}
I20260811 09:19:23.221433 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.199904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:54769 (local address 127.17.16.65:36239)
0811 09:19:23.212045 (+ 12141us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:23.212051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:23.212077 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:19:23.212144 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:23.212149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:23.212306 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:23.212944 (+ 638us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:23.213225 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.214509 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.216993 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.219256 (+ 2263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.220768 (+ 1512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.220824 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.220879 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.221035 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11990,"spinlock_wait_cycles":45312,"thread_start_us":11809,"threads_started":1}
I20260811 09:19:23.221374 16507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d91636701304e139ed73178179a1a2b" candidate_uuid: "c448b5572a5a46148f2adb22e6302934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "958e2eb5924a435287416dd1fefd31ef" is_pre_election: true
I20260811 09:19:23.221549 16941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d91636701304e139ed73178179a1a2b" candidate_uuid: "c448b5572a5a46148f2adb22e6302934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2510ddaf9fda4f59a9d639c5394bb318" is_pre_election: true
I20260811 09:19:23.221633 16731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d91636701304e139ed73178179a1a2b" candidate_uuid: "2510ddaf9fda4f59a9d639c5394bb318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c448b5572a5a46148f2adb22e6302934" is_pre_election: true
I20260811 09:19:23.221957 16941 raft_consensus.cc:2454] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c448b5572a5a46148f2adb22e6302934 in term 0.
I20260811 09:19:23.222041 16507 raft_consensus.cc:2454] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c448b5572a5a46148f2adb22e6302934 in term 0.
I20260811 09:19:23.222098 16731 raft_consensus.cc:2454] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2510ddaf9fda4f59a9d639c5394bb318 in term 0.
I20260811 09:19:23.222352 16504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d91636701304e139ed73178179a1a2b" candidate_uuid: "2510ddaf9fda4f59a9d639c5394bb318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "958e2eb5924a435287416dd1fefd31ef" is_pre_election: true
I20260811 09:19:23.222652 16504 raft_consensus.cc:2454] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2510ddaf9fda4f59a9d639c5394bb318 in term 0.
I20260811 09:19:23.222733 16620 leader_election.cc:302] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2510ddaf9fda4f59a9d639c5394bb318, c448b5572a5a46148f2adb22e6302934; no voters:
I20260811 09:19:23.222887 16830 leader_election.cc:302] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2510ddaf9fda4f59a9d639c5394bb318, c448b5572a5a46148f2adb22e6302934; no voters:
I20260811 09:19:23.223189 17052 raft_consensus.cc:2781] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:19:23.223349 17052 raft_consensus.cc:482] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:19:23.223369 17055 raft_consensus.cc:2781] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:19:23.223474 17052 raft_consensus.cc:3037] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:23.223487 17055 raft_consensus.cc:482] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:19:23.223596 17055 raft_consensus.cc:3037] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:23.226904 17055 raft_consensus.cc:504] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.226904 17052 raft_consensus.cc:504] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.227591 17052 leader_election.cc:288] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [CANDIDATE]: Term 1 election: Requested vote from peers 2510ddaf9fda4f59a9d639c5394bb318 (127.17.16.67:43709), 958e2eb5924a435287416dd1fefd31ef (127.17.16.65:36239)
I20260811 09:19:23.227591 17055 leader_election.cc:288] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [CANDIDATE]: Term 1 election: Requested vote from peers 958e2eb5924a435287416dd1fefd31ef (127.17.16.65:36239), c448b5572a5a46148f2adb22e6302934 (127.17.16.66:35513)
I20260811 09:19:23.228121 16507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d91636701304e139ed73178179a1a2b" candidate_uuid: "2510ddaf9fda4f59a9d639c5394bb318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "958e2eb5924a435287416dd1fefd31ef"
I20260811 09:19:23.228173 16504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d91636701304e139ed73178179a1a2b" candidate_uuid: "c448b5572a5a46148f2adb22e6302934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "958e2eb5924a435287416dd1fefd31ef"
I20260811 09:19:23.228255 16941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d91636701304e139ed73178179a1a2b" candidate_uuid: "c448b5572a5a46148f2adb22e6302934" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2510ddaf9fda4f59a9d639c5394bb318"
I20260811 09:19:23.228340 16507 raft_consensus.cc:3037] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:23.228340 16731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d91636701304e139ed73178179a1a2b" candidate_uuid: "2510ddaf9fda4f59a9d639c5394bb318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c448b5572a5a46148f2adb22e6302934"
I20260811 09:19:23.228518 16941 raft_consensus.cc:2379] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c448b5572a5a46148f2adb22e6302934 in current term 1: Already voted for candidate 2510ddaf9fda4f59a9d639c5394bb318 in this term.
I20260811 09:19:23.228582 16731 raft_consensus.cc:2379] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2510ddaf9fda4f59a9d639c5394bb318 in current term 1: Already voted for candidate c448b5572a5a46148f2adb22e6302934 in this term.
I20260811 09:19:23.229300 16620 leader_election.cc:302] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c448b5572a5a46148f2adb22e6302934; no voters: 2510ddaf9fda4f59a9d639c5394bb318, 958e2eb5924a435287416dd1fefd31ef
I20260811 09:19:23.229584 17052 raft_consensus.cc:2726] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 09:19:23.230862 16507 raft_consensus.cc:2454] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2510ddaf9fda4f59a9d639c5394bb318 in term 1.
I20260811 09:19:23.231204 16831 leader_election.cc:302] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2510ddaf9fda4f59a9d639c5394bb318, 958e2eb5924a435287416dd1fefd31ef; no voters: c448b5572a5a46148f2adb22e6302934
I20260811 09:19:23.231504 17055 raft_consensus.cc:2781] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:23.231935 17055 raft_consensus.cc:686] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 1 LEADER]: Becoming Leader. State: Replica: 2510ddaf9fda4f59a9d639c5394bb318, State: Running, Role: LEADER
I20260811 09:19:23.232378 17055 consensus_queue.cc:237] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } }
I20260811 09:19:23.236812 16271 catalog_manager.cc:5495] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2510ddaf9fda4f59a9d639c5394bb318 (127.17.16.67). New cstate: current_term: 1 leader_uuid: "2510ddaf9fda4f59a9d639c5394bb318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2510ddaf9fda4f59a9d639c5394bb318" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 43709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 } health_report { overall_health: UNKNOWN } } }
W20260811 09:19:23.240806 16587 tablet.cc:2369] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:19:23.278075 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.272815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.17.16.67:43709)
0811 09:19:23.273021 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:23.273025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:23.273048 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:19:23.273249 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:23.273253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:23.273355 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:23.273477 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:19:23.273792 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.274589 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.275868 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.276283 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.276371 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.276922 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:19:23.277417 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:19:23.277538 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.277726 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.277773 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 09:19:23.286089 16507 raft_consensus.cc:1261] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [term 1 FOLLOWER]: Refusing update from remote peer 2510ddaf9fda4f59a9d639c5394bb318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:23.286118 16731 raft_consensus.cc:1261] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 1 FOLLOWER]: Refusing update from remote peer 2510ddaf9fda4f59a9d639c5394bb318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:23.286835 17055 consensus_queue.cc:1035] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c448b5572a5a46148f2adb22e6302934" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:23.287158 17061 consensus_queue.cc:1035] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "958e2eb5924a435287416dd1fefd31ef" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 36239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:23.300034 17067 mvcc.cc:205] Tried to move back new op lower bound from 7317258089609674752 to 7317258089403678720. Current Snapshot: MvccSnapshot[applied={T|T < 7317258089609674752}]
I20260811 09:19:23.301084 17066 mvcc.cc:205] Tried to move back new op lower bound from 7317258089609674752 to 7317258089403678720. Current Snapshot: MvccSnapshot[applied={T|T < 7317258089609674752}]
I20260811 09:19:23.314239 16272 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:23.314548 16272 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:23.318222 16272 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 928d11b894aa4b9e916a0a13da804f94: Sending DeleteTablet for 3 replicas of tablet 0d91636701304e139ed73178179a1a2b
I20260811 09:19:23.319250 16907 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d91636701304e139ed73178179a1a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:23 UTC) from {username='slave'} at 127.0.0.1:41392
I20260811 09:19:23.319305 16702 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d91636701304e139ed73178179a1a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:23 UTC) from {username='slave'} at 127.0.0.1:38826
I20260811 09:19:23.319324 16472 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d91636701304e139ed73178179a1a2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:23 UTC) from {username='slave'} at 127.0.0.1:34950
I20260811 09:19:23.320223 17083 tablet_replica.cc:330] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: stopping tablet replica
I20260811 09:19:23.320674 17083 raft_consensus.cc:2229] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:23.321204 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 16365
I20260811 09:19:23.321275 17083 raft_consensus.cc:2258] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:23.331110 17085 tablet_replica.cc:330] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef: stopping tablet replica
I20260811 09:19:23.331722 17085 raft_consensus.cc:2229] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:23.332311 17085 raft_consensus.cc:2258] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:23.333341 17083 ts_tablet_manager.cc:1905] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:23.334511 17085 ts_tablet_manager.cc:1905] T 0d91636701304e139ed73178179a1a2b P 958e2eb5924a435287416dd1fefd31ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:23.336799 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.321955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34956 (local address 127.17.16.65:36239)
0811 09:19:23.322239 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:23.322245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:23.324798 (+ 2553us) server_negotiation.cc:407] Connection header received
0811 09:19:23.325096 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:23.325102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:23.325230 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:23.325308 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:23.325966 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.327087 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.329024 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.329685 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.329784 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.335146 (+ 5362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:23.335158 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:23.335175 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:23.335903 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:23.335964 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:23.335990 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:23.336077 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.336341 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.336463 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260811 09:19:23.338224 16578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:23.338876 17081 tablet_replica.cc:330] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: stopping tablet replica
I20260811 09:19:23.339372 17081 raft_consensus.cc:2229] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:19:23.340314 17081 raft_consensus.cc:2258] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:23.343510 17083 ts_tablet_manager.cc:1918] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:23.344280 17081 ts_tablet_manager.cc:1905] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:23.349181 17083 log.cc:1196] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/0d91636701304e139ed73178179a1a2b
I20260811 09:19:23.349284 17081 ts_tablet_manager.cc:1918] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:23.349447 17081 log.cc:1196] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/0d91636701304e139ed73178179a1a2b
I20260811 09:19:23.349998 17081 ts_tablet_manager.cc:1939] T 0d91636701304e139ed73178179a1a2b P 2510ddaf9fda4f59a9d639c5394bb318: Deleting consensus metadata
I20260811 09:19:23.349999 17083 ts_tablet_manager.cc:1939] T 0d91636701304e139ed73178179a1a2b P c448b5572a5a46148f2adb22e6302934: Deleting consensus metadata
I20260811 09:19:23.352005 16257 catalog_manager.cc:4841] TS 2510ddaf9fda4f59a9d639c5394bb318 (127.17.16.67:43709): tablet 0d91636701304e139ed73178179a1a2b (table test-table [id=a85d3c5002214785aac7a78898cae5af]) successfully deleted
I20260811 09:19:23.351971 16245 catalog_manager.cc:4841] TS c448b5572a5a46148f2adb22e6302934 (127.17.16.66:35513): tablet 0d91636701304e139ed73178179a1a2b (table test-table [id=a85d3c5002214785aac7a78898cae5af]) successfully deleted
I20260811 09:19:23.476794 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 16365
W20260811 09:19:23.501498 16249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.16.65:36239 (error 108)
I20260811 09:19:23.502887 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 16590
I20260811 09:19:23.503496 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.502995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.0.0.1:34958)
0811 09:19:23.503204 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.503336 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:36239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 09:19:23.503916 16249 catalog_manager.cc:4584] TS 958e2eb5924a435287416dd1fefd31ef (127.17.16.65:36239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0d91636701304e139ed73178179a1a2b: Network error: Client connection negotiation failed: client connection to 127.17.16.65:36239: connect: Connection refused (error 111)
I20260811 09:19:23.510705 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.503653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38832 (local address 127.17.16.66:35513)
0811 09:19:23.503818 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:23.503824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:23.503905 (+ 81us) server_negotiation.cc:407] Connection header received
0811 09:19:23.504117 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:23.504123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:23.504250 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:23.504333 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:23.504777 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.505725 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.507461 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.507920 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.508011 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.509213 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:23.509225 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:23.509240 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:23.509875 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:23.509928 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:23.509969 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:23.510070 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.510218 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.510326 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:19:23.511435 16794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:23.557577 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.557013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.0.0.1:34974)
0811 09:19:23.557255 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.557368 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:36239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 09:19:23.558795 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.558366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.0.0.1:34980)
0811 09:19:23.558536 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.558625 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:36239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 09:19:23.601678 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 16590
I20260811 09:19:23.619679 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.619114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.0.0.1:34988)
0811 09:19:23.619366 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.619473 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:36239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 09:19:23.620961 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.620475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.0.0.1:34990)
0811 09:19:23.620665 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.620766 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:36239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 09:19:23.627851 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 16803
I20260811 09:19:23.635500 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.628578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.17.16.67:43709)
0811 09:19:23.628803 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:23.628808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:23.628831 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:19:23.629016 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:23.629022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:23.629137 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:23.629257 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:23.629647 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.630786 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.632034 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:23.632731 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:23.632866 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:23.633833 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:23.633845 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:23.633865 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:23.634767 (+ 902us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:23.634831 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:23.634860 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:23.634981 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:23.635052 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 09:19:23.635148 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 09:19:23.636224 17009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:23.702147 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.701609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.0.0.1:34994)
0811 09:19:23.701832 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.701983 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:36239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 09:19:23.703236 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.702864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.0.0.1:35008)
0811 09:19:23.703045 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.703117 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:36239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 09:19:23.757855 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 16803
I20260811 09:19:23.784759 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 16222
I20260811 09:19:23.786031 16321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:23.893947 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:23.893199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:36239 (local address 127.0.0.1:35024)
0811 09:19:23.893570 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:23.893706 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:36239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 09:19:23.894011 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 16222
2026-08-11T09:19:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3022 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:24 dist-test-slave-ncp9 krb5kdc[17101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:24 dist-test-slave-ncp9 krb5kdc[17101](info): set up 2 sockets
Aug 11 09:19:24 dist-test-slave-ncp9 krb5kdc[17101](info): commencing operation
krb5kdc: starting...
W20260811 09:19:25.146654 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:25 dist-test-slave-ncp9 krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:19:25.468242 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41919
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:25.805013 17122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:25.805325 17122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:25.805469 17122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:25.812747 17122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:25.812873 17122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:25.812969 17122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:25.813057 17122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:25.813125 17122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:25.823017 17122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:25.824363 17122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:25.825716 17122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:25.837110 17134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:25.837198 17136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:25.837221 17138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:25.837984 17122 server_base.cc:1034] running on GCE node
I20260811 09:19:25.838735 17122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:25.840049 17122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:25.841243 17122 hybrid_clock.cc:648] HybridClock initialized: now 1786439965841199 us; error 32 us; skew 500 ppm
I20260811 09:19:25.841576 17122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:25 dist-test-slave-ncp9 krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439965, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:25.844555 17122 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:19:25.845971 17122 webserver.cc:460] Webserver started at http://127.17.16.126:39689/ using document root <none> and password file <none>
I20260811 09:19:25.846624 17122 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:25.846741 17122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:25.847038 17122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:25.850013 17122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "cc915a0b8b714bd8bcd19bae09bb1873"
format_stamp: "Formatted at 2026-08-11 09:19:25 on dist-test-slave-ncp9"
server_key: "0e4f91c567a9725d1ac700c37b98655a"
server_key_iv: "3b3b826da14774742fba0b6333fdea42"
server_key_version: "encryptionkey@0"
I20260811 09:19:25.850744 17122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "cc915a0b8b714bd8bcd19bae09bb1873"
format_stamp: "Formatted at 2026-08-11 09:19:25 on dist-test-slave-ncp9"
server_key: "0e4f91c567a9725d1ac700c37b98655a"
server_key_iv: "3b3b826da14774742fba0b6333fdea42"
server_key_version: "encryptionkey@0"
I20260811 09:19:25.855775 17122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:19:25.859234 17155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:25.860456 17122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:19:25.860607 17122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "cc915a0b8b714bd8bcd19bae09bb1873"
format_stamp: "Formatted at 2026-08-11 09:19:25 on dist-test-slave-ncp9"
server_key: "0e4f91c567a9725d1ac700c37b98655a"
server_key_iv: "3b3b826da14774742fba0b6333fdea42"
server_key_version: "encryptionkey@0"
I20260811 09:19:25.860782 17122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:25.873437 17122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:25.877578 17122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:25.877842 17122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:25.895115 17122 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:41919
I20260811 09:19:25.895123 17233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:41919 every 8 connection(s)
I20260811 09:19:25.896463 17122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:25.900096 17234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:25.906363 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 17122
I20260811 09:19:25.906561 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:25.907099 17473 external_mini_cluster.cc:1428] Setting key 2465bbef4d83587730ed2ae951b24f70
I20260811 09:19:25.910995 17234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873: Bootstrap starting.
I20260811 09:19:25.914683 17234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:25.915733 17234 log.cc:824] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:25.918758 17234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873: No bootstrap required, opened a new log
Aug 11 09:19:25 dist-test-slave-ncp9 krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:25.927923 17234 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc915a0b8b714bd8bcd19bae09bb1873" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41919 } }
I20260811 09:19:25.928400 17234 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:25.928524 17234 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc915a0b8b714bd8bcd19bae09bb1873, State: Initialized, Role: FOLLOWER
I20260811 09:19:25.929220 17234 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc915a0b8b714bd8bcd19bae09bb1873" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41919 } }
I20260811 09:19:25.929473 17234 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:25.929617 17234 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:25.929790 17234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:25.930083 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:25.909054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42004 (local address 127.17.16.126:41919)
0811 09:19:25.909812 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:25.909823 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:25.909865 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:19:25.911580 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:25.911603 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:25.911972 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:25.912316 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:25.914141 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:25.915465 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:25.917095 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:25.917755 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:25.917876 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:25.922699 (+ 4823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:25.922707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:25.922722 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:25.922761 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:25.925857 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:25.926712 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:25.926719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:25.926729 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:25.926893 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:25.927273 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:25.927277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:25.927279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:25.927794 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:25.928009 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:25.928532 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 09:19:25.928987 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":363,"threads_started":1}
I20260811 09:19:25.932214 17234 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc915a0b8b714bd8bcd19bae09bb1873" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41919 } }
I20260811 09:19:25.932723 17234 leader_election.cc:302] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc915a0b8b714bd8bcd19bae09bb1873; no voters:
I20260811 09:19:25.933271 17234 leader_election.cc:288] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:25.933385 17242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:25.934032 17242 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [term 1 LEADER]: Becoming Leader. State: Replica: cc915a0b8b714bd8bcd19bae09bb1873, State: Running, Role: LEADER
I20260811 09:19:25.934580 17242 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc915a0b8b714bd8bcd19bae09bb1873" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41919 } }
I20260811 09:19:25.935264 17234 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:25.937757 17244 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc915a0b8b714bd8bcd19bae09bb1873. Latest consensus state: current_term: 1 leader_uuid: "cc915a0b8b714bd8bcd19bae09bb1873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc915a0b8b714bd8bcd19bae09bb1873" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41919 } } }
I20260811 09:19:25.938174 17244 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:25.940078 17243 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc915a0b8b714bd8bcd19bae09bb1873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc915a0b8b714bd8bcd19bae09bb1873" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41919 } } }
I20260811 09:19:25.940447 17243 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:25.940770 17248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:25.945837 17248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:25.955356 17248 catalog_manager.cc:1347] Generated new cluster ID: ac3887433ec9458c8ab64b728b12c52d
I20260811 09:19:25.955461 17248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:25.965894 17248 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:25.967784 17248 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:25.975636 17248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873: Generated new TSK 0
I20260811 09:19:25.976442 17248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.16.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
Entry for principal kudu/127.17.16.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab.
I20260811 09:19:26.063488 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41919
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:26.440934 17271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:26.441212 17271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:26.441316 17271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:26.448541 17271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:26.448649 17271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:26.448768 17271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:19:26.458539 17271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.65.keytab
--principal=kudu/127.17.16.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:26.459789 17271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:26.461081 17271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:26.473459 17280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:26.473512 17281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:26.473520 17283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:26.474299 17271 server_base.cc:1034] running on GCE node
I20260811 09:19:26.474881 17271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:26.475775 17271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:26.476977 17271 hybrid_clock.cc:648] HybridClock initialized: now 1786439966476920 us; error 53 us; skew 500 ppm
I20260811 09:19:26.477277 17271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:26 dist-test-slave-ncp9 krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439966, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:26.480237 17271 init.cc:377] Logged in from keytab as kudu/127.17.16.65@KRBTEST.COM (short username kudu)
I20260811 09:19:26.481612 17271 webserver.cc:460] Webserver started at http://127.17.16.65:36685/ using document root <none> and password file <none>
I20260811 09:19:26.482287 17271 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:26.482403 17271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:26.482714 17271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:26.485620 17271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "4e75ef775c1743e0ac29f5e9b0731104"
format_stamp: "Formatted at 2026-08-11 09:19:26 on dist-test-slave-ncp9"
server_key: "7a073bb6385bbb644d1fcb37dfe1701d"
server_key_iv: "df6c6b53e7f2cc3c757b4cc3ecddc8ee"
server_key_version: "encryptionkey@0"
I20260811 09:19:26.486379 17271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "4e75ef775c1743e0ac29f5e9b0731104"
format_stamp: "Formatted at 2026-08-11 09:19:26 on dist-test-slave-ncp9"
server_key: "7a073bb6385bbb644d1fcb37dfe1701d"
server_key_iv: "df6c6b53e7f2cc3c757b4cc3ecddc8ee"
server_key_version: "encryptionkey@0"
I20260811 09:19:26.491555 17271 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:19:26.495002 17299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:26.496312 17271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:19:26.496459 17271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "4e75ef775c1743e0ac29f5e9b0731104"
format_stamp: "Formatted at 2026-08-11 09:19:26 on dist-test-slave-ncp9"
server_key: "7a073bb6385bbb644d1fcb37dfe1701d"
server_key_iv: "df6c6b53e7f2cc3c757b4cc3ecddc8ee"
server_key_version: "encryptionkey@0"
I20260811 09:19:26.496646 17271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:26.509886 17271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:26.513837 17271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:26.514082 17271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:26.514955 17271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:26.516218 17271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:26.516290 17271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:26.516377 17271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:26.516423 17271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:26.546967 17271 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:46219
I20260811 09:19:26.547003 17490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:46219 every 8 connection(s)
I20260811 09:19:26.548234 17271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:19:26.551144 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 17271
I20260811 09:19:26.551337 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:19:26.551909 17473 external_mini_cluster.cc:1428] Setting key 502d119c1271914e6735e11df5cb5a37
Aug 11 09:19:26 dist-test-slave-ncp9 krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439966, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.65@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:26.567653 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:26.550692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:56385 (local address 127.17.16.126:41919)
0811 09:19:26.551161 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:26.551165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:26.552184 (+ 1019us) server_negotiation.cc:407] Connection header received
0811 09:19:26.554441 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:26.554446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:26.554524 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:26.554617 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:26.556069 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:26.556796 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:26.558323 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:26.558736 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:26.558848 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:26.562254 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:26.562259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:26.562263 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:26.562293 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:26.564743 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:26.565472 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:26.565476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:26.565478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:26.565601 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:26.566013 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:26.566017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:26.566018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:26.566269 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:26.566397 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:26.567076 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 09:19:26.567294 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":221,"threads_started":1}
I20260811 09:19:26.568583 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:26.551275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41919 (local address 127.17.16.65:56385)
0811 09:19:26.552036 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:26.552080 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:26.554198 (+ 2118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:26.554847 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:26.554855 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:26.555357 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:26.555900 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:26.555916 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:26.556968 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:26.556972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:26.558186 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:26.558199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:26.558949 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:26.558958 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:26.559197 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:26.559812 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:19:26.559834 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:26.562047 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:26.564913 (+ 2866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:26.564923 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:26.564938 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:26.565352 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:26.565705 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:26.565710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:26.565714 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:26.565900 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:26.566396 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:26.566403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:26.566714 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 09:19:26.566935 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 09:19:26.567437 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":305,"threads_started":1}
I20260811 09:19:26.570093 17491 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41919
I20260811 09:19:26.570382 17491 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:26.571050 17491 heartbeater.cc:511] Master 127.17.16.126:41919 requested a full tablet report, sending...
I20260811 09:19:26.573273 17177 ts_manager.cc:194] Registered new tserver with Master: 4e75ef775c1743e0ac29f5e9b0731104 (127.17.16.65:46219)
I20260811 09:19:26.575133 17177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.65@KRBTEST.COM'} at 127.17.16.65:56385
WARNING: no policy specified for kudu/127.17.16.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
Entry for principal kudu/127.17.16.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab.
I20260811 09:19:26.618045 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41919
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:26.950888 17499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:26.951177 17499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:26.951300 17499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:26.958493 17499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:26.958606 17499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:26.958760 17499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:19:26.968851 17499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.66.keytab
--principal=kudu/127.17.16.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:26.970160 17499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:26.971410 17499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:26.983981 17509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:26.984105 17505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:26.984081 17506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:26.985122 17499 server_base.cc:1034] running on GCE node
I20260811 09:19:26.985780 17499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:26.986696 17499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:26.987895 17499 hybrid_clock.cc:648] HybridClock initialized: now 1786439966987843 us; error 43 us; skew 500 ppm
I20260811 09:19:26.988196 17499 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:26 dist-test-slave-ncp9 krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439966, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:26.991214 17499 init.cc:377] Logged in from keytab as kudu/127.17.16.66@KRBTEST.COM (short username kudu)
I20260811 09:19:26.992596 17499 webserver.cc:460] Webserver started at http://127.17.16.66:45077/ using document root <none> and password file <none>
I20260811 09:19:26.993304 17499 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:26.993425 17499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:26.993738 17499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:26.996713 17499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "0d8b4e6404b648709bc228122839f929"
format_stamp: "Formatted at 2026-08-11 09:19:26 on dist-test-slave-ncp9"
server_key: "8726dc9d948a188d56c11d2a4372a324"
server_key_iv: "f052cc9557bfaabe7555e7967408f7ab"
server_key_version: "encryptionkey@0"
I20260811 09:19:26.997500 17499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "0d8b4e6404b648709bc228122839f929"
format_stamp: "Formatted at 2026-08-11 09:19:26 on dist-test-slave-ncp9"
server_key: "8726dc9d948a188d56c11d2a4372a324"
server_key_iv: "f052cc9557bfaabe7555e7967408f7ab"
server_key_version: "encryptionkey@0"
I20260811 09:19:27.002619 17499 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:19:27.005939 17520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:27.007174 17499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:19:27.007324 17499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "0d8b4e6404b648709bc228122839f929"
format_stamp: "Formatted at 2026-08-11 09:19:26 on dist-test-slave-ncp9"
server_key: "8726dc9d948a188d56c11d2a4372a324"
server_key_iv: "f052cc9557bfaabe7555e7967408f7ab"
server_key_version: "encryptionkey@0"
I20260811 09:19:27.007517 17499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:27.019354 17499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:27.023259 17499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:27.023524 17499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:27.024392 17499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:27.025656 17499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:27.025732 17499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:27.025846 17499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:27.025898 17499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:27.056143 17499 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:35929
I20260811 09:19:27.056190 17702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:35929 every 8 connection(s)
I20260811 09:19:27.057394 17499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:19:27.066916 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 17499
I20260811 09:19:27.067116 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:19:27.067698 17473 external_mini_cluster.cc:1428] Setting key ad0cf6b7bea032a77ceb37006958890e
Aug 11 09:19:27 dist-test-slave-ncp9 krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439966, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.66@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:27.076795 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.059824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:52395 (local address 127.17.16.126:41919)
0811 09:19:27.060056 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:27.060062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:27.061199 (+ 1137us) server_negotiation.cc:407] Connection header received
0811 09:19:27.063004 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:27.063009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:27.063082 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:27.063169 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:27.064588 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.065343 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.066923 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.067557 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.067681 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.071415 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:27.071421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:27.071424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:27.071456 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:27.073958 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:27.074740 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:27.074744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:27.074746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:27.074865 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:27.075281 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:27.075284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:27.075286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:27.075537 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:27.075652 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:27.076402 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 09:19:27.076565 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:19:27.078254 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.060318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41919 (local address 127.17.16.66:52395)
0811 09:19:27.061035 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:27.061078 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:27.062738 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:27.063351 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:27.063359 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:27.063867 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:27.064447 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.064465 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.065474 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.065478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.066777 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.066795 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.067762 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.067769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.067986 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.068669 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:19:27.068701 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:27.071193 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:27.074089 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:27.074097 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:27.074111 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:27.074647 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:27.074956 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:27.074960 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:27.074962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:27.075192 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:27.075649 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:27.075656 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:27.076052 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 09:19:27.076331 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 09:19:27.076933 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":273,"threads_started":1}
I20260811 09:19:27.079756 17703 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41919
I20260811 09:19:27.080098 17703 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:27.081046 17703 heartbeater.cc:511] Master 127.17.16.126:41919 requested a full tablet report, sending...
I20260811 09:19:27.082604 17177 ts_manager.cc:194] Registered new tserver with Master: 0d8b4e6404b648709bc228122839f929 (127.17.16.66:35929)
I20260811 09:19:27.083562 17177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.66@KRBTEST.COM'} at 127.17.16.66:52395
WARNING: no policy specified for kudu/127.17.16.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
Entry for principal kudu/127.17.16.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab.
I20260811 09:19:27.136682 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41919
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:27.469730 17712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:27.470000 17712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:27.470144 17712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:27.477205 17712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:27.477322 17712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:27.477474 17712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:19:27.486927 17712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.67.keytab
--principal=kudu/127.17.16.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:27.488189 17712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:27.489481 17712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:27.501837 17718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:27.501868 17722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:27.501905 17719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:27.502955 17712 server_base.cc:1034] running on GCE node
I20260811 09:19:27.503536 17712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:27.504384 17712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:27.505584 17712 hybrid_clock.cc:648] HybridClock initialized: now 1786439967505529 us; error 47 us; skew 500 ppm
I20260811 09:19:27.505896 17712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:27 dist-test-slave-ncp9 krb5kdc[17101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439967, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:27.508991 17712 init.cc:377] Logged in from keytab as kudu/127.17.16.67@KRBTEST.COM (short username kudu)
I20260811 09:19:27.511921 17712 webserver.cc:460] Webserver started at http://127.17.16.67:38871/ using document root <none> and password file <none>
I20260811 09:19:27.512635 17712 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:27.512740 17712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:27.513051 17712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:27.516054 17712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "c73ab86531084913b7eeb1da3d62d739"
format_stamp: "Formatted at 2026-08-11 09:19:27 on dist-test-slave-ncp9"
server_key: "e241457082d8414d83396ddb53c7e9b6"
server_key_iv: "f54f1c61956d724b90d961dcd44174c8"
server_key_version: "encryptionkey@0"
I20260811 09:19:27.516800 17712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "c73ab86531084913b7eeb1da3d62d739"
format_stamp: "Formatted at 2026-08-11 09:19:27 on dist-test-slave-ncp9"
server_key: "e241457082d8414d83396ddb53c7e9b6"
server_key_iv: "f54f1c61956d724b90d961dcd44174c8"
server_key_version: "encryptionkey@0"
I20260811 09:19:27.521934 17712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:19:27.525281 17736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:27.526504 17712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:19:27.526657 17712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "c73ab86531084913b7eeb1da3d62d739"
format_stamp: "Formatted at 2026-08-11 09:19:27 on dist-test-slave-ncp9"
server_key: "e241457082d8414d83396ddb53c7e9b6"
server_key_iv: "f54f1c61956d724b90d961dcd44174c8"
server_key_version: "encryptionkey@0"
I20260811 09:19:27.526826 17712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:27.542055 17712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:27.546162 17712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:27.546474 17712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:27.547606 17712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:27.549233 17712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:27.549326 17712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:27.549453 17712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:27.549513 17712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:27.578042 17491 heartbeater.cc:503] Master 127.17.16.126:41919 was elected leader, sending a full tablet report...
I20260811 09:19:27.580650 17712 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:42123
I20260811 09:19:27.580687 17920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:42123 every 8 connection(s)
I20260811 09:19:27.581907 17712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:19:27.586093 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 17712
I20260811 09:19:27.586289 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:19:27.586866 17473 external_mini_cluster.cc:1428] Setting key c86b6f5aa8f26b67a91347f179edc39c
Aug 11 09:19:27 dist-test-slave-ncp9 krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439967, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.67@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:27.600978 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.584184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:35971 (local address 127.17.16.126:41919)
0811 09:19:27.584661 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:27.584668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:27.585700 (+ 1032us) server_negotiation.cc:407] Connection header received
0811 09:19:27.588034 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:27.588039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:27.588122 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:27.588221 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:27.590006 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.590786 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.592212 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.592595 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.592680 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.595886 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:27.595891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:27.595895 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:27.595926 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:27.598233 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:27.598824 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:27.598827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:27.598830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:27.598940 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:27.599285 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:27.599288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:27.599290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:27.599537 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:27.599648 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:27.600423 (+ 775us) server_negotiation.cc:299] Negotiation successful
0811 09:19:27.600611 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":218,"threads_started":1}
I20260811 09:19:27.601974 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.584770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41919 (local address 127.17.16.67:35971)
0811 09:19:27.585541 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:27.585585 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:27.587782 (+ 2197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:27.588509 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:27.588519 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:27.589173 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:27.589856 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.589877 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.590894 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.590897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.592095 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.592107 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.592752 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.592756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.593002 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.593584 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:19:27.593606 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:27.595698 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:27.598343 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:27.598349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:27.598360 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:27.598736 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:27.599009 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:19:27.599012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:19:27.599014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:19:27.599192 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:19:27.599621 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:27.599626 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:27.599931 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 09:19:27.600231 (+ 300us) client_negotiation.cc:240] Negotiation successful
0811 09:19:27.600692 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":293,"threads_started":1}
I20260811 09:19:27.603128 17921 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41919
I20260811 09:19:27.603392 17921 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:27.604085 17921 heartbeater.cc:511] Master 127.17.16.126:41919 requested a full tablet report, sending...
I20260811 09:19:27.605530 17177 ts_manager.cc:194] Registered new tserver with Master: c73ab86531084913b7eeb1da3d62d739 (127.17.16.67:42123)
I20260811 09:19:27.606348 17177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.16.67@KRBTEST.COM'} at 127.17.16.67:35971
I20260811 09:19:27.613986 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:19:27.629945 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.618038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.17.16.126:41919)
0811 09:19:27.618212 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:27.618219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:27.618641 (+ 422us) server_negotiation.cc:407] Connection header received
0811 09:19:27.618754 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:27.618758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:27.618818 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:27.618892 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:27.620204 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.620972 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.622177 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.622564 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.622662 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.625208 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:27.625213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:27.625217 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:27.625247 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:27.627648 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:27.628267 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:27.628271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:27.628273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:27.628379 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:27.628670 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:27.628673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:27.628675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:27.629016 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:27.629200 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:27.629492 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 09:19:27.629688 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:19:27.633572 17177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:19:27.640532 17177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:19:27.663025 17947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.652680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.17.16.65:46219)
0811 09:19:27.653289 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:27.653297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:27.654397 (+ 1100us) server_negotiation.cc:407] Connection header received
0811 09:19:27.654901 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:27.654910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:27.655143 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:27.655342 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:27.656689 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.658062 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.660946 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.661893 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.662019 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.662275 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:27.662552 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 09:19:27.662716 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":293,"threads_started":1}
I20260811 09:19:27.663090 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.652446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:46219 (local address 127.0.0.1:38240)
0811 09:19:27.653962 (+ 1516us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:27.654015 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:27.654629 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:27.655360 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:27.655368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:27.655408 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:27.656500 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.656519 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.658185 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.658191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.660733 (+ 2542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.660749 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.662093 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.662100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.662327 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.662365 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 09:19:27.662473 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 09:19:27.662665 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1253,"mutex_wait_us":41,"thread_start_us":1122,"threads_started":1}
I20260811 09:19:27.663930 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.653160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.17.16.67:42123)
0811 09:19:27.654856 (+ 1696us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:27.654863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:27.654892 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:19:27.654974 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:27.654980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:27.655154 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:27.655341 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:27.656785 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.658247 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.660961 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.662411 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.662540 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.662756 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:27.663450 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 09:19:27.663598 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1546,"thread_start_us":1387,"threads_started":1}
I20260811 09:19:27.663928 17956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.654746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.17.16.66:35929)
0811 09:19:27.656568 (+ 1822us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:27.656576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:27.656602 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:19:27.656707 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:27.656714 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:27.656960 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:27.657149 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:27.657671 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.658867 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.660996 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.662411 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.663202 (+ 791us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.663377 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:27.663439 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 09:19:27.663605 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1658,"thread_start_us":1542,"threads_started":1}
I20260811 09:19:27.664475 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.652620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:42123 (local address 127.0.0.1:44562)
0811 09:19:27.654010 (+ 1390us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:27.654023 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:27.654188 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:27.655619 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:27.655624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:27.655658 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:27.656577 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.656592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.658386 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.658391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.660744 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.660757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.662898 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.662904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.663097 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.663128 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:19:27.663207 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 09:19:27.663338 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1298,"mutex_wait_us":78,"thread_start_us":1454,"threads_started":1}
I20260811 09:19:27.663842 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.654432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35929 (local address 127.0.0.1:47114)
0811 09:19:27.655982 (+ 1550us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:27.656000 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:27.656102 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:27.657155 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:27.657160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:27.657186 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:27.657501 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.657511 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.659006 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.659012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.660843 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.660859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.662654 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.662660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.662823 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.662860 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 09:19:27.662926 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 09:19:27.663055 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1435,"thread_start_us":1262,"threads_started":1}
I20260811 09:19:27.667968 17812 tablet_service.cc:1467] Processing CreateTablet for tablet 44efac16e5a7461db302a68f4d03ba50 (DEFAULT_TABLE table=test-table [id=1f85ea6931434191896429ef32a81269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:27.667927 17384 tablet_service.cc:1467] Processing CreateTablet for tablet 44efac16e5a7461db302a68f4d03ba50 (DEFAULT_TABLE table=test-table [id=1f85ea6931434191896429ef32a81269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:27.667922 17593 tablet_service.cc:1467] Processing CreateTablet for tablet 44efac16e5a7461db302a68f4d03ba50 (DEFAULT_TABLE table=test-table [id=1f85ea6931434191896429ef32a81269]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:27.669267 17593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44efac16e5a7461db302a68f4d03ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:27.669348 17384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44efac16e5a7461db302a68f4d03ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:27.669363 17812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44efac16e5a7461db302a68f4d03ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:27.680198 17957 tablet_bootstrap.cc:492] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: Bootstrap starting.
I20260811 09:19:27.683638 17958 tablet_bootstrap.cc:492] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: Bootstrap starting.
I20260811 09:19:27.683696 17957 tablet_bootstrap.cc:654] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:27.684439 17959 tablet_bootstrap.cc:492] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: Bootstrap starting.
I20260811 09:19:27.684989 17957 log.cc:824] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:27.687578 17958 tablet_bootstrap.cc:654] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:27.687872 17959 tablet_bootstrap.cc:654] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:27.688738 17958 log.cc:824] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:27.688937 17959 log.cc:824] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:27.690444 17957 tablet_bootstrap.cc:492] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: No bootstrap required, opened a new log
I20260811 09:19:27.690748 17957 ts_tablet_manager.cc:1397] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260811 09:19:27.691074 17958 tablet_bootstrap.cc:492] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: No bootstrap required, opened a new log
I20260811 09:19:27.691298 17958 ts_tablet_manager.cc:1397] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260811 09:19:27.696861 17959 tablet_bootstrap.cc:492] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: No bootstrap required, opened a new log
I20260811 09:19:27.697176 17959 ts_tablet_manager.cc:1397] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260811 09:19:27.698779 17958 raft_consensus.cc:348] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 } } peers { permanent_uuid: "c73ab86531084913b7eeb1da3d62d739" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42123 } } peers { permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 } }
I20260811 09:19:27.699117 17958 raft_consensus.cc:374] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:27.699244 17958 raft_consensus.cc:729] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e75ef775c1743e0ac29f5e9b0731104, State: Initialized, Role: FOLLOWER
I20260811 09:19:27.699813 17958 consensus_queue.cc:260] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 } } peers { permanent_uuid: "c73ab86531084913b7eeb1da3d62d739" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42123 } } peers { permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 } }
I20260811 09:19:27.701648 17958 ts_tablet_manager.cc:1428] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: Time spent starting tablet: real 0.010s user 0.005s sys 0.006s
I20260811 09:19:27.701376 17957 raft_consensus.cc:348] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 } } peers { permanent_uuid: "c73ab86531084913b7eeb1da3d62d739" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42123 } } peers { permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 } }
I20260811 09:19:27.701802 17957 raft_consensus.cc:374] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:27.701918 17957 raft_consensus.cc:729] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d8b4e6404b648709bc228122839f929, State: Initialized, Role: FOLLOWER
I20260811 09:19:27.702548 17957 consensus_queue.cc:260] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 } } peers { permanent_uuid: "c73ab86531084913b7eeb1da3d62d739" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42123 } } peers { permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 } }
I20260811 09:19:27.703642 17703 heartbeater.cc:503] Master 127.17.16.126:41919 was elected leader, sending a full tablet report...
I20260811 09:19:27.704277 17957 ts_tablet_manager.cc:1428] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: Time spent starting tablet: real 0.013s user 0.007s sys 0.006s
I20260811 09:19:27.708606 17959 raft_consensus.cc:348] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 } } peers { permanent_uuid: "c73ab86531084913b7eeb1da3d62d739" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42123 } } peers { permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 } }
I20260811 09:19:27.709084 17959 raft_consensus.cc:374] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:27.709177 17959 raft_consensus.cc:729] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c73ab86531084913b7eeb1da3d62d739, State: Initialized, Role: FOLLOWER
I20260811 09:19:27.709686 17959 consensus_queue.cc:260] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 } } peers { permanent_uuid: "c73ab86531084913b7eeb1da3d62d739" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42123 } } peers { permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 } }
I20260811 09:19:27.710626 17921 heartbeater.cc:503] Master 127.17.16.126:41919 was elected leader, sending a full tablet report...
I20260811 09:19:27.711212 17959 ts_tablet_manager.cc:1428] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: Time spent starting tablet: real 0.014s user 0.008s sys 0.003s
W20260811 09:19:27.800457 17492 tablet.cc:2369] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:19:27.809288 17704 tablet.cc:2369] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:19:27.833710 17925 tablet.cc:2369] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:19:27.840595 17970 raft_consensus.cc:482] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:19:27.840929 17970 raft_consensus.cc:504] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 } } peers { permanent_uuid: "c73ab86531084913b7eeb1da3d62d739" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42123 } } peers { permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 } }
I20260811 09:19:27.842752 17970 leader_election.cc:288] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e75ef775c1743e0ac29f5e9b0731104 (127.17.16.65:46219), 0d8b4e6404b648709bc228122839f929 (127.17.16.66:35929)
I20260811 09:19:27.849390 17956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.843587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:45767 (local address 127.17.16.66:35929)
0811 09:19:27.843828 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:27.843834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:27.843853 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:19:27.843934 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:27.843939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:27.844015 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:27.844109 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:27.844611 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.845622 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.847435 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.848535 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.848650 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.848704 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:27.849089 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 09:19:27.849197 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:19:27.849488 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.843059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35929 (local address 127.17.16.67:45767)
0811 09:19:27.843341 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:27.843361 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:27.843494 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:27.844205 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:27.844214 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:27.844247 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:27.844429 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.844438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.845751 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.845757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.847277 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.847290 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.848684 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.848690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.848885 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.848956 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 09:19:27.849075 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 09:19:27.849221 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260811 09:19:27.850405 17947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.843224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:51473 (local address 127.17.16.65:46219)
0811 09:19:27.843422 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:27.843428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:27.844327 (+ 899us) server_negotiation.cc:407] Connection header received
0811 09:19:27.844598 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:27.844604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:27.844670 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:27.844748 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:27.845360 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.846291 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.848253 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.849402 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.849563 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.849639 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:27.850045 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 09:19:27.850176 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:19:27.850440 17971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.843151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:46219 (local address 127.17.16.67:51473)
0811 09:19:27.844230 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:27.844247 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:27.844435 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:27.844818 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:27.844823 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:27.844850 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:27.845209 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.845221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.846440 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.846443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.848077 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:27.848092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.849746 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.849752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:27.849894 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.849917 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:19:27.849999 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 09:19:27.850122 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"thread_start_us":716,"threads_started":1}
I20260811 09:19:27.850560 17624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44efac16e5a7461db302a68f4d03ba50" candidate_uuid: "c73ab86531084913b7eeb1da3d62d739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d8b4e6404b648709bc228122839f929" is_pre_election: true
I20260811 09:19:27.851159 17624 raft_consensus.cc:2454] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c73ab86531084913b7eeb1da3d62d739 in term 0.
I20260811 09:19:27.851426 17415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44efac16e5a7461db302a68f4d03ba50" candidate_uuid: "c73ab86531084913b7eeb1da3d62d739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e75ef775c1743e0ac29f5e9b0731104" is_pre_election: true
I20260811 09:19:27.851816 17738 leader_election.cc:302] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d8b4e6404b648709bc228122839f929, c73ab86531084913b7eeb1da3d62d739; no voters:
I20260811 09:19:27.851866 17415 raft_consensus.cc:2454] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c73ab86531084913b7eeb1da3d62d739 in term 0.
I20260811 09:19:27.852212 17970 raft_consensus.cc:2781] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:19:27.852376 17970 raft_consensus.cc:482] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:19:27.852494 17970 raft_consensus.cc:3037] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:27.854717 17970 raft_consensus.cc:504] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 } } peers { permanent_uuid: "c73ab86531084913b7eeb1da3d62d739" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42123 } } peers { permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 } }
I20260811 09:19:27.855304 17970 leader_election.cc:288] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [CANDIDATE]: Term 1 election: Requested vote from peers 4e75ef775c1743e0ac29f5e9b0731104 (127.17.16.65:46219), 0d8b4e6404b648709bc228122839f929 (127.17.16.66:35929)
I20260811 09:19:27.855639 17415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44efac16e5a7461db302a68f4d03ba50" candidate_uuid: "c73ab86531084913b7eeb1da3d62d739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e75ef775c1743e0ac29f5e9b0731104"
I20260811 09:19:27.855819 17415 raft_consensus.cc:3037] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:27.855800 17624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44efac16e5a7461db302a68f4d03ba50" candidate_uuid: "c73ab86531084913b7eeb1da3d62d739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d8b4e6404b648709bc228122839f929"
I20260811 09:19:27.855970 17624 raft_consensus.cc:3037] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:27.858011 17415 raft_consensus.cc:2454] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c73ab86531084913b7eeb1da3d62d739 in term 1.
I20260811 09:19:27.858305 17624 raft_consensus.cc:2454] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c73ab86531084913b7eeb1da3d62d739 in term 1.
I20260811 09:19:27.858372 17737 leader_election.cc:302] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e75ef775c1743e0ac29f5e9b0731104, c73ab86531084913b7eeb1da3d62d739; no voters:
I20260811 09:19:27.858657 17970 raft_consensus.cc:2781] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:27.859071 17970 raft_consensus.cc:686] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 1 LEADER]: Becoming Leader. State: Replica: c73ab86531084913b7eeb1da3d62d739, State: Running, Role: LEADER
I20260811 09:19:27.859524 17970 consensus_queue.cc:237] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 } } peers { permanent_uuid: "c73ab86531084913b7eeb1da3d62d739" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42123 } } peers { permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 } }
I20260811 09:19:27.864245 17177 catalog_manager.cc:5495] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 reported cstate change: term changed from 0 to 1, leader changed from <none> to c73ab86531084913b7eeb1da3d62d739 (127.17.16.67). New cstate: current_term: 1 leader_uuid: "c73ab86531084913b7eeb1da3d62d739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c73ab86531084913b7eeb1da3d62d739" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 42123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 } health_report { overall_health: UNKNOWN } } }
I20260811 09:19:27.903395 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.897143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44570 (local address 127.17.16.67:42123)
0811 09:19:27.897373 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:27.897379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:27.897401 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:19:27.897629 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:27.897633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:27.897717 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:27.897866 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:19:27.898275 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.899029 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.900571 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.901098 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.901232 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.901775 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:19:27.902468 (+ 693us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:19:27.902637 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:27.903100 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 09:19:27.903189 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 09:19:27.911563 17415 raft_consensus.cc:1261] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104 [term 1 FOLLOWER]: Refusing update from remote peer c73ab86531084913b7eeb1da3d62d739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:27.911578 17624 raft_consensus.cc:1261] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929 [term 1 FOLLOWER]: Refusing update from remote peer c73ab86531084913b7eeb1da3d62d739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:27.912515 17970 consensus_queue.cc:1035] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e75ef775c1743e0ac29f5e9b0731104" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 46219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:27.912786 17972 consensus_queue.cc:1035] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d8b4e6404b648709bc228122839f929" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:27.928270 17979 mvcc.cc:205] Tried to move back new op lower bound from 7317258108555767808 to 7317258108356014080. Current Snapshot: MvccSnapshot[applied={T|T < 7317258108555767808}]
I20260811 09:19:27.938067 17177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:27.938373 17177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:27.941972 17177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc915a0b8b714bd8bcd19bae09bb1873: Sending DeleteTablet for 3 replicas of tablet 44efac16e5a7461db302a68f4d03ba50
I20260811 09:19:27.943038 17384 tablet_service.cc:1514] Processing DeleteTablet for tablet 44efac16e5a7461db302a68f4d03ba50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:27 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:38240
I20260811 09:19:27.943313 17812 tablet_service.cc:1514] Processing DeleteTablet for tablet 44efac16e5a7461db302a68f4d03ba50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:27 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:44562
I20260811 09:19:27.943435 17593 tablet_service.cc:1514] Processing DeleteTablet for tablet 44efac16e5a7461db302a68f4d03ba50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:27 UTC) from {username='kudu', principal='kudu/127.17.16.126@KRBTEST.COM'} at 127.0.0.1:47114
I20260811 09:19:27.947018 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 17271
I20260811 09:19:27.961591 17992 tablet_replica.cc:330] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: stopping tablet replica
I20260811 09:19:27.961831 17990 tablet_replica.cc:330] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: stopping tablet replica
I20260811 09:19:27.962040 17992 raft_consensus.cc:2229] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:27.962320 17990 raft_consensus.cc:2229] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:27.962622 17992 raft_consensus.cc:2258] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:27.962855 17990 raft_consensus.cc:2258] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:27.965691 17991 tablet_replica.cc:330] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: stopping tablet replica
I20260811 09:19:27.966135 17991 raft_consensus.cc:2229] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:19:27.966843 17991 raft_consensus.cc:2258] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:27.966934 17992 ts_tablet_manager.cc:1905] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:27.967649 17990 ts_tablet_manager.cc:1905] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:27.968926 17991 ts_tablet_manager.cc:1905] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:19:27 dist-test-slave-ncp9 krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.65@KRBTEST.COM
I20260811 09:19:27.974715 17992 ts_tablet_manager.cc:1918] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:27.975028 17992 log.cc:1196] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/44efac16e5a7461db302a68f4d03ba50
I20260811 09:19:27.975620 17992 ts_tablet_manager.cc:1939] T 44efac16e5a7461db302a68f4d03ba50 P 0d8b4e6404b648709bc228122839f929: Deleting consensus metadata
I20260811 09:19:27.976470 17991 ts_tablet_manager.cc:1918] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:27.976681 17991 log.cc:1196] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/44efac16e5a7461db302a68f4d03ba50
I20260811 09:19:27.977274 17991 ts_tablet_manager.cc:1939] T 44efac16e5a7461db302a68f4d03ba50 P c73ab86531084913b7eeb1da3d62d739: Deleting consensus metadata
I20260811 09:19:27.977841 17157 catalog_manager.cc:4841] TS 0d8b4e6404b648709bc228122839f929 (127.17.16.66:35929): tablet 44efac16e5a7461db302a68f4d03ba50 (table test-table [id=1f85ea6931434191896429ef32a81269]) successfully deleted
I20260811 09:19:27.978914 17990 ts_tablet_manager.cc:1918] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:27.978960 17157 catalog_manager.cc:4841] TS c73ab86531084913b7eeb1da3d62d739 (127.17.16.67:42123): tablet 44efac16e5a7461db302a68f4d03ba50 (table test-table [id=1f85ea6931434191896429ef32a81269]) successfully deleted
I20260811 09:19:27.979079 17990 log.cc:1196] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439649204149-17473-0/minicluster-data/ts-0/wal/wals/44efac16e5a7461db302a68f4d03ba50
I20260811 09:19:27.979547 17990 ts_tablet_manager.cc:1939] T 44efac16e5a7461db302a68f4d03ba50 P 4e75ef775c1743e0ac29f5e9b0731104: Deleting consensus metadata
I20260811 09:19:27.979605 17947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:27.948480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38252 (local address 127.17.16.65:46219)
0811 09:19:27.948667 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:27.948674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:27.958459 (+ 9785us) server_negotiation.cc:407] Connection header received
0811 09:19:27.958971 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:27.958978 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:27.959073 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:27.959157 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:27.963032 (+ 3875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.964026 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.965679 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:27.966345 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:27.966447 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:27.971985 (+ 5538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:27.972002 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:27.972020 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:27.972063 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:27.975735 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:27.976692 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:27.976703 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:27.976706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:27.976959 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:27.977468 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:27.977474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:27.977477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:27.977992 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:27.978295 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:27.978927 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 09:19:27.979234 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:19:27.980782 17484 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:27.981140 17156 catalog_manager.cc:4841] TS 4e75ef775c1743e0ac29f5e9b0731104 (127.17.16.65:46219): tablet 44efac16e5a7461db302a68f4d03ba50 (table test-table [id=1f85ea6931434191896429ef32a81269]) successfully deleted
I20260811 09:19:28.073405 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 17271
I20260811 09:19:28.097115 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 17499
Aug 11 09:19:28 dist-test-slave-ncp9 krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.66@KRBTEST.COM
I20260811 09:19:28.113672 17956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:28.097875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.17.16.66:35929)
0811 09:19:28.098076 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:28.098082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:28.098104 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:19:28.098194 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:28.098199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:28.098281 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:28.098383 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:28.099595 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:28.100751 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:28.102185 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:28.102809 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:28.102933 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:28.107122 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:28.107140 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:28.107161 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:28.107203 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:28.110880 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:28.111530 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:28.111541 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:28.111545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:28.111695 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:28.112024 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:28.112037 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:28.112041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:28.112535 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:28.112842 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:28.113065 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 09:19:28.113288 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:19:28.114485 17698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:28.219097 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 17499
I20260811 09:19:28.244652 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 17712
Aug 11 09:19:28 dist-test-slave-ncp9 krb5kdc[17101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.67@KRBTEST.COM
I20260811 09:19:28.264251 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:28.245576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44584 (local address 127.17.16.67:42123)
0811 09:19:28.245731 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:28.245735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:28.245760 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:19:28.245925 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:28.245932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:28.246011 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:28.246096 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:28.247901 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:28.248753 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:28.250686 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:28.251384 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:28.251507 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:28.257233 (+ 5726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:28.257249 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:28.257271 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:28.257337 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:28.261247 (+ 3910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:28.261913 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:28.261923 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:28.261927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:28.262155 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:28.262504 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:28.262510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:28.262514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:28.263040 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:28.263405 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:28.263572 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 09:19:28.263807 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 09:19:28.265085 17912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:28.384435 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 17712
I20260811 09:19:28.407917 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 17122
I20260811 09:19:28.409096 17227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:28.518848 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 17122
2026-08-11T09:19:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4622 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T09:19:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:28Z Disabled control of system clock
I20260811 09:19:28.610579 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41737
--webserver_interface=127.17.16.126
--webserver_port=0
--builtin_ntp_servers=127.17.16.84:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:19:28.946415 18009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:28.946821 18009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:28.946961 18009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:28.956830 18009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:28.957000 18009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:28.957077 18009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:28.957190 18009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:28.957248 18009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:28.966626 18009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41737
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:28.967891 18009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:28.969686 18009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:28.981009 18022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:28.981082 18024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:28.981110 18028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:28.981822 18009 server_base.cc:1034] running on GCE node
I20260811 09:19:28.982579 18009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:28.983907 18009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:28.985097 18009 hybrid_clock.cc:648] HybridClock initialized: now 1786439968985038 us; error 48 us; skew 500 ppm
I20260811 09:19:28.985417 18009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:28.991470 18009 webserver.cc:460] Webserver started at http://127.17.16.126:34431/ using document root <none> and password file <none>
I20260811 09:19:28.992147 18009 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:28.992252 18009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:28.992568 18009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:28.995450 18009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "b2a200eb1df64e4594d5e3c2bb0accca"
format_stamp: "Formatted at 2026-08-11 09:19:28 on dist-test-slave-ncp9"
server_key: "8e41f1540501a8afbd4309a72b589a72"
server_key_iv: "81ddae4ff8b324e73995500c7facd6b2"
server_key_version: "encryptionkey@0"
I20260811 09:19:28.996264 18009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "b2a200eb1df64e4594d5e3c2bb0accca"
format_stamp: "Formatted at 2026-08-11 09:19:28 on dist-test-slave-ncp9"
server_key: "8e41f1540501a8afbd4309a72b589a72"
server_key_iv: "81ddae4ff8b324e73995500c7facd6b2"
server_key_version: "encryptionkey@0"
I20260811 09:19:29.001422 18009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 09:19:29.004871 18036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:29.006191 18009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:19:29.006358 18009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "b2a200eb1df64e4594d5e3c2bb0accca"
format_stamp: "Formatted at 2026-08-11 09:19:28 on dist-test-slave-ncp9"
server_key: "8e41f1540501a8afbd4309a72b589a72"
server_key_iv: "81ddae4ff8b324e73995500c7facd6b2"
server_key_version: "encryptionkey@0"
I20260811 09:19:29.006570 18009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:29.051859 18009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:29.052790 18009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:29.053100 18009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:29.070549 18009 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:41737
I20260811 09:19:29.070571 18118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:41737 every 8 connection(s)
I20260811 09:19:29.071888 18009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:29.075567 18119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:29.078081 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 18009
I20260811 09:19:29.078233 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:29.078655 17473 external_mini_cluster.cc:1428] Setting key a46bdb7e2f2b82859769238d0172b058
I20260811 09:19:29.088706 18119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca: Bootstrap starting.
I20260811 09:19:29.092542 18119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:29.093928 18119 log.cc:824] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:29.094370 18122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:29.080640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.17.16.126:41737)
0811 09:19:29.081715 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:29.081729 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:29.081780 (+ 51us) server_negotiation.cc:407] Connection header received
0811 09:19:29.084055 (+ 2275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:29.084097 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:29.084728 (+ 631us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:29.085218 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:29.085755 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:29.087738 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:29.089110 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:29.090124 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:29.090332 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:29.091399 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:29.091409 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:29.091425 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:29.092106 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:29.092220 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:29.092247 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:29.092364 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:29.092641 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 09:19:29.093080 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":343,"threads_started":1}
I20260811 09:19:29.097100 18119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca: No bootstrap required, opened a new log
I20260811 09:19:29.104249 18119 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a200eb1df64e4594d5e3c2bb0accca" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41737 } }
I20260811 09:19:29.104581 18119 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:29.104696 18119 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a200eb1df64e4594d5e3c2bb0accca, State: Initialized, Role: FOLLOWER
I20260811 09:19:29.105350 18119 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a200eb1df64e4594d5e3c2bb0accca" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41737 } }
I20260811 09:19:29.105620 18119 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:29.105768 18119 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:29.105926 18119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:29.108016 18119 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a200eb1df64e4594d5e3c2bb0accca" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41737 } }
I20260811 09:19:29.108493 18119 leader_election.cc:302] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2a200eb1df64e4594d5e3c2bb0accca; no voters:
I20260811 09:19:29.109102 18119 leader_election.cc:288] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:29.109229 18125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:29.109985 18125 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [term 1 LEADER]: Becoming Leader. State: Replica: b2a200eb1df64e4594d5e3c2bb0accca, State: Running, Role: LEADER
I20260811 09:19:29.110625 18125 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a200eb1df64e4594d5e3c2bb0accca" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41737 } }
I20260811 09:19:29.111348 18119 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:29.113641 18126 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2a200eb1df64e4594d5e3c2bb0accca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a200eb1df64e4594d5e3c2bb0accca" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41737 } } }
I20260811 09:19:29.114049 18126 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:29.117045 18129 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a200eb1df64e4594d5e3c2bb0accca. Latest consensus state: current_term: 1 leader_uuid: "b2a200eb1df64e4594d5e3c2bb0accca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a200eb1df64e4594d5e3c2bb0accca" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41737 } } }
I20260811 09:19:29.117893 18129 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:29.117942 18136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:29.123320 18136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:29.133170 18136 catalog_manager.cc:1347] Generated new cluster ID: a2f425f87b7c489f94b6a80723e8f2fe
I20260811 09:19:29.133287 18136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:29.146790 18136 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:29.148070 18136 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:29.153855 18136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca: Generated new TSK 0
I20260811 09:19:29.154675 18136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:29.165488 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.65:0
--local_ip_for_outbound_sockets=127.17.16.65
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41737
--builtin_ntp_servers=127.17.16.84:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:19:29.514576 18161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:29.514882 18161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:29.515026 18161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:29.522714 18161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:29.522819 18161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:29.522933 18161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.65
I20260811 09:19:29.532095 18161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.16.65
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.65
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:29.533409 18161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:29.535166 18161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:29.547915 18167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:29.547971 18168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:29.548089 18170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:29.548943 18161 server_base.cc:1034] running on GCE node
I20260811 09:19:29.549500 18161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:29.550386 18161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:29.551594 18161 hybrid_clock.cc:648] HybridClock initialized: now 1786439969551533 us; error 52 us; skew 500 ppm
I20260811 09:19:29.551901 18161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:29.554152 18161 webserver.cc:460] Webserver started at http://127.17.16.65:45439/ using document root <none> and password file <none>
I20260811 09:19:29.554821 18161 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:29.554924 18161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:29.555238 18161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:29.558148 18161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/data/instance:
uuid: "1f181f28de2d498cb806c848dd50a05d"
format_stamp: "Formatted at 2026-08-11 09:19:29 on dist-test-slave-ncp9"
server_key: "d42f6e4798bf3609f724985e9a91684f"
server_key_iv: "94a164772bb1821f976113be9065aacf"
server_key_version: "encryptionkey@0"
I20260811 09:19:29.558925 18161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance:
uuid: "1f181f28de2d498cb806c848dd50a05d"
format_stamp: "Formatted at 2026-08-11 09:19:29 on dist-test-slave-ncp9"
server_key: "d42f6e4798bf3609f724985e9a91684f"
server_key_iv: "94a164772bb1821f976113be9065aacf"
server_key_version: "encryptionkey@0"
I20260811 09:19:29.563880 18161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 09:19:29.567364 18184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:29.568650 18161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:19:29.568797 18161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/wal
uuid: "1f181f28de2d498cb806c848dd50a05d"
format_stamp: "Formatted at 2026-08-11 09:19:29 on dist-test-slave-ncp9"
server_key: "d42f6e4798bf3609f724985e9a91684f"
server_key_iv: "94a164772bb1821f976113be9065aacf"
server_key_version: "encryptionkey@0"
I20260811 09:19:29.569036 18161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:29.582154 18161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:29.582979 18161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:29.583289 18161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:29.584250 18161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:29.585762 18161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:29.585860 18161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:29.585979 18161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:29.586050 18161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:29.617132 18161 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.65:44845
I20260811 09:19:29.617177 18373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.65:44845 every 8 connection(s)
I20260811 09:19:29.618579 18161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/data/info.pb
I20260811 09:19:29.623703 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 18161
I20260811 09:19:29.623862 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-0/wal/instance
I20260811 09:19:29.624396 17473 external_mini_cluster.cc:1428] Setting key fe05446db2951c23dd0eb274b0bb4265
I20260811 09:19:29.627213 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.66:0
--local_ip_for_outbound_sockets=127.17.16.66
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41737
--builtin_ntp_servers=127.17.16.84:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:19:29.631512 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:29.621425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.65:59959 (local address 127.17.16.126:41737)
0811 09:19:29.621895 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:29.621900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:29.623023 (+ 1123us) server_negotiation.cc:407] Connection header received
0811 09:19:29.624937 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:29.624944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:29.625031 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:29.625137 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:29.626474 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:29.627185 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:29.628772 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:29.629236 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:29.629353 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:29.630686 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:29.630692 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:29.630695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:29.630713 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:29.630755 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:29.630767 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:29.630841 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:29.631058 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 09:19:29.631162 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":258,"threads_started":1}
I20260811 09:19:29.632553 18380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:29.621894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41737 (local address 127.17.16.65:59959)
0811 09:19:29.622843 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:29.622882 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:29.624648 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:29.625412 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:29.625424 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:29.625541 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:29.626305 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:29.626327 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:29.627333 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:29.627339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:29.628629 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:29.628643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:29.629468 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:29.629474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:29.629713 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:29.630382 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:19:29.630384 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:29.630426 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:19:29.630507 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:29.630856 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:29.630869 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:29.630876 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 09:19:29.631035 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 09:19:29.631452 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":314,"threads_started":1}
I20260811 09:19:29.634035 18374 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41737
I20260811 09:19:29.634359 18374 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:29.635262 18374 heartbeater.cc:511] Master 127.17.16.126:41737 requested a full tablet report, sending...
I20260811 09:19:29.637457 18065 ts_manager.cc:194] Registered new tserver with Master: 1f181f28de2d498cb806c848dd50a05d (127.17.16.65:44845)
I20260811 09:19:29.639015 18065 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.65:59959
W20260811 09:19:29.965553 18381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:29.965924 18381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:29.966060 18381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:29.974401 18381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:29.974514 18381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:29.974653 18381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.66
I20260811 09:19:29.983728 18381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.16.66
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.66
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:29.985021 18381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:29.986806 18381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:29.999619 18396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:29.999682 18391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:29.999707 18392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:30.000716 18381 server_base.cc:1034] running on GCE node
I20260811 09:19:30.001386 18381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:30.002333 18381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:30.003564 18381 hybrid_clock.cc:648] HybridClock initialized: now 1786439970003500 us; error 55 us; skew 500 ppm
I20260811 09:19:30.003878 18381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:30.006150 18381 webserver.cc:460] Webserver started at http://127.17.16.66:34249/ using document root <none> and password file <none>
I20260811 09:19:30.006819 18381 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:30.006916 18381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:30.007223 18381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:30.010160 18381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/data/instance:
uuid: "0189bcdf97b14bf3a397e122c779db28"
format_stamp: "Formatted at 2026-08-11 09:19:30 on dist-test-slave-ncp9"
server_key: "c649f8077aaea2a9da3e0ee23e496740"
server_key_iv: "54175a486801565d4bca4fdb2d7d87b7"
server_key_version: "encryptionkey@0"
I20260811 09:19:30.010946 18381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance:
uuid: "0189bcdf97b14bf3a397e122c779db28"
format_stamp: "Formatted at 2026-08-11 09:19:30 on dist-test-slave-ncp9"
server_key: "c649f8077aaea2a9da3e0ee23e496740"
server_key_iv: "54175a486801565d4bca4fdb2d7d87b7"
server_key_version: "encryptionkey@0"
I20260811 09:19:30.015959 18381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:19:30.019338 18407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:30.020649 18381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:19:30.020774 18381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/wal
uuid: "0189bcdf97b14bf3a397e122c779db28"
format_stamp: "Formatted at 2026-08-11 09:19:30 on dist-test-slave-ncp9"
server_key: "c649f8077aaea2a9da3e0ee23e496740"
server_key_iv: "54175a486801565d4bca4fdb2d7d87b7"
server_key_version: "encryptionkey@0"
I20260811 09:19:30.021006 18381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:30.036139 18381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:30.036934 18381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:30.037210 18381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:30.038133 18381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:30.039588 18381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:30.039671 18381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:30.039754 18381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:30.039808 18381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:30.073880 18381 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.66:35503
I20260811 09:19:30.073971 18588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.66:35503 every 8 connection(s)
I20260811 09:19:30.075183 18381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/data/info.pb
I20260811 09:19:30.080209 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 18381
I20260811 09:19:30.080392 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/wal/instance
I20260811 09:19:30.080968 17473 external_mini_cluster.cc:1428] Setting key ec63d22d50848883f01424c814634d6a
I20260811 09:19:30.084311 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.16.67:0
--local_ip_for_outbound_sockets=127.17.16.67
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41737
--builtin_ntp_servers=127.17.16.84:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:19:30.093971 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.083498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.66:33371 (local address 127.17.16.126:41737)
0811 09:19:30.083671 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.083677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.084855 (+ 1178us) server_negotiation.cc:407] Connection header received
0811 09:19:30.087286 (+ 2431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.087290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.087355 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.087442 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:30.088689 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.089456 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.091420 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.091793 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.091879 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.093322 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:30.093327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:30.093330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:30.093347 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:30.093379 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:30.093391 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:30.093457 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.093676 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.093753 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:19:30.095275 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.083619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41737 (local address 127.17.16.66:33371)
0811 09:19:30.084710 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:30.084762 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:30.086976 (+ 2214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:30.087688 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:30.087699 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:30.087805 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:30.088538 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.088560 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.089600 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.089605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.091302 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.091317 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.091998 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.092005 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.092272 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.092991 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:19:30.092994 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:30.093052 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:19:30.093148 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:30.093476 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:30.093485 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:30.093493 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:19:30.093654 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 09:19:30.094080 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":320,"threads_started":1}
I20260811 09:19:30.096577 18591 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41737
I20260811 09:19:30.096876 18591 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:30.097766 18591 heartbeater.cc:511] Master 127.17.16.126:41737 requested a full tablet report, sending...
I20260811 09:19:30.099291 18065 ts_manager.cc:194] Registered new tserver with Master: 0189bcdf97b14bf3a397e122c779db28 (127.17.16.66:35503)
I20260811 09:19:30.100322 18065 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.66:33371
W20260811 09:19:30.421368 18596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:30.421648 18596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:30.421783 18596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:30.428857 18596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:30.428983 18596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:30.429131 18596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.16.67
I20260811 09:19:30.437955 18596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.16.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.16.67
--webserver_port=0
--tserver_master_addrs=127.17.16.126:41737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.16.67
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:30.439214 18596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:30.440974 18596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:30.453456 18605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:30.453456 18610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:30.453504 18606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:30.454586 18596 server_base.cc:1034] running on GCE node
I20260811 09:19:30.455109 18596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:30.455925 18596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:30.457111 18596 hybrid_clock.cc:648] HybridClock initialized: now 1786439970457076 us; error 29 us; skew 500 ppm
I20260811 09:19:30.457376 18596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:19:30.459374 18596 webserver.cc:460] Webserver started at http://127.17.16.67:35451/ using document root <none> and password file <none>
I20260811 09:19:30.459995 18596 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:30.460078 18596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:30.460327 18596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:30.463056 18596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/data/instance:
uuid: "6835b23ae1e54c0bb09a87534735b956"
format_stamp: "Formatted at 2026-08-11 09:19:30 on dist-test-slave-ncp9"
server_key: "7a412383eecd7e9ae56ab944833b1804"
server_key_iv: "a0bea1659e251ce79720f01a5b0b37ae"
server_key_version: "encryptionkey@0"
I20260811 09:19:30.463757 18596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance:
uuid: "6835b23ae1e54c0bb09a87534735b956"
format_stamp: "Formatted at 2026-08-11 09:19:30 on dist-test-slave-ncp9"
server_key: "7a412383eecd7e9ae56ab944833b1804"
server_key_iv: "a0bea1659e251ce79720f01a5b0b37ae"
server_key_version: "encryptionkey@0"
I20260811 09:19:30.468437 18596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 09:19:30.471602 18619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:30.472937 18596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:19:30.473078 18596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/wal
uuid: "6835b23ae1e54c0bb09a87534735b956"
format_stamp: "Formatted at 2026-08-11 09:19:30 on dist-test-slave-ncp9"
server_key: "7a412383eecd7e9ae56ab944833b1804"
server_key_iv: "a0bea1659e251ce79720f01a5b0b37ae"
server_key_version: "encryptionkey@0"
I20260811 09:19:30.473275 18596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:30.491990 18596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:30.492741 18596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:30.493047 18596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:30.493924 18596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:19:30.495326 18596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:19:30.495410 18596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:30.495487 18596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:19:30.495550 18596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:30.524875 18596 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.67:40415
I20260811 09:19:30.525024 18801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.67:40415 every 8 connection(s)
I20260811 09:19:30.526198 18596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/data/info.pb
I20260811 09:19:30.527541 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 18596
I20260811 09:19:30.527678 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/wal/instance
I20260811 09:19:30.528072 17473 external_mini_cluster.cc:1428] Setting key 506b09a9c4e754b0cf40936ea911322e
I20260811 09:19:30.538966 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.529847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:48673 (local address 127.17.16.126:41737)
0811 09:19:30.530015 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.530023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.531629 (+ 1606us) server_negotiation.cc:407] Connection header received
0811 09:19:30.533352 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.533357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.533423 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.533495 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:30.534428 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.535165 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.536614 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.537089 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.537191 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.538366 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:30.538371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:30.538374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:30.538391 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:30.538419 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:30.538430 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:30.538494 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.538659 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.538732 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:19:30.539914 18805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.530450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41737 (local address 127.17.16.67:48673)
0811 09:19:30.531461 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:30.531505 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:30.533102 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:30.533670 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:30.533678 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:30.533757 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:19:30.534305 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.534322 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.535265 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.535268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.536471 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.536483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.537267 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.537273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.537475 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.538094 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:19:30.538097 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:19:30.538144 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:19:30.538221 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:19:30.538502 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:19:30.538508 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:19:30.538513 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 09:19:30.538640 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 09:19:30.538907 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":318,"threads_started":1}
I20260811 09:19:30.541184 18802 heartbeater.cc:347] Connected to a master server at 127.17.16.126:41737
I20260811 09:19:30.541445 18802 heartbeater.cc:464] Registering TS with master...
I20260811 09:19:30.542131 18802 heartbeater.cc:511] Master 127.17.16.126:41737 requested a full tablet report, sending...
I20260811 09:19:30.543573 18065 ts_manager.cc:194] Registered new tserver with Master: 6835b23ae1e54c0bb09a87534735b956 (127.17.16.67:40415)
I20260811 09:19:30.544363 18065 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.16.67:48673
I20260811 09:19:30.554823 17473 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:19:30.564144 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.558779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.17.16.126:41737)
0811 09:19:30.558916 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.558920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.559401 (+ 481us) server_negotiation.cc:407] Connection header received
0811 09:19:30.559521 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.559525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.559602 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.559698 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:30.560134 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.560840 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.562112 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.562531 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.562632 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.563576 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:30.563581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:30.563585 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:30.563604 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:30.563638 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:30.563652 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:30.563725 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.563816 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.563904 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 09:19:30.567612 18065 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:19:30.574494 18065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:19:30.626348 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.586572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54978 (local address 127.17.16.66:35503)
0811 09:19:30.616494 (+ 29922us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.616504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.617212 (+ 708us) server_negotiation.cc:407] Connection header received
0811 09:19:30.617538 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.617547 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.617837 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.618015 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:30.619306 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.620960 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.623766 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.625174 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.625300 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.625541 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.625841 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.626017 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29742,"thread_start_us":29928,"threads_started":1}
I20260811 09:19:30.626412 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.586274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35503 (local address 127.0.0.1:54978)
0811 09:19:30.617079 (+ 30805us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:30.617100 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:30.617333 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:30.618694 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:30.618701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:30.618734 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:30.619106 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.619120 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.621165 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.621170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.623504 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.623529 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.625383 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.625390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.625594 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.625650 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 09:19:30.625791 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 09:19:30.625998 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30690,"mutex_wait_us":4,"thread_start_us":30542,"threads_started":1}
I20260811 09:19:30.627085 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.586034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.67:40415 (local address 127.0.0.1:43534)
0811 09:19:30.595307 (+ 9273us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:30.595344 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:30.595525 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:30.619378 (+ 23853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:30.619382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:30.619407 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:30.619811 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.619841 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.622163 (+ 2322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.622168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.624336 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.624352 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.626053 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.626060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.626258 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.626293 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:19:30.626363 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 09:19:30.626485 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8957,"mutex_wait_us":52,"thread_start_us":32109,"threads_started":1}
I20260811 09:19:30.627460 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.586281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43534 (local address 127.17.16.67:40415)
0811 09:19:30.617537 (+ 31256us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.617545 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.617581 (+ 36us) server_negotiation.cc:407] Connection header received
0811 09:19:30.617666 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.617672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.617839 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.620080 (+ 2241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:30.620415 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.622012 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.624519 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.625842 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.626616 (+ 774us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.626869 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.626940 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.627108 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31081,"thread_start_us":30918,"threads_started":1}
I20260811 09:19:30.630041 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.587209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:44845 (local address 127.0.0.1:44244)
0811 09:19:30.618422 (+ 31213us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:30.618441 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:30.618647 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:30.619712 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:30.619716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:30.619743 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:30.620175 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.620190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.622423 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.622428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.626005 (+ 3577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.626033 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.628109 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.628115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.629478 (+ 1363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.629511 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:19:30.629610 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 09:19:30.629734 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31087,"mutex_wait_us":3,"thread_start_us":7651,"threads_started":1}
I20260811 09:19:30.630770 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.587708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.17.16.65:44845)
0811 09:19:30.615852 (+ 28144us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.615862 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.618538 (+ 2676us) server_negotiation.cc:407] Connection header received
0811 09:19:30.618890 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.618899 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.619155 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.619381 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:30.620406 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.622228 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.626260 (+ 4032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.627596 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.627738 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.627975 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.630267 (+ 2292us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.630433 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27923,"thread_start_us":28161,"threads_started":1}
I20260811 09:19:30.630846 18686 tablet_service.cc:1467] Processing CreateTablet for tablet 6531f95d8ac34b82a40fc8fc5b47e2f1 (DEFAULT_TABLE table=test-table [id=df7d475bde0941f2b156d0dd8d2ed75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:30.630831 18475 tablet_service.cc:1467] Processing CreateTablet for tablet 6531f95d8ac34b82a40fc8fc5b47e2f1 (DEFAULT_TABLE table=test-table [id=df7d475bde0941f2b156d0dd8d2ed75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:30.632179 18475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6531f95d8ac34b82a40fc8fc5b47e2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:30.632268 18686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6531f95d8ac34b82a40fc8fc5b47e2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:30.634348 18269 tablet_service.cc:1467] Processing CreateTablet for tablet 6531f95d8ac34b82a40fc8fc5b47e2f1 (DEFAULT_TABLE table=test-table [id=df7d475bde0941f2b156d0dd8d2ed75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:19:30.635604 18269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6531f95d8ac34b82a40fc8fc5b47e2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:30.642155 18374 heartbeater.cc:503] Master 127.17.16.126:41737 was elected leader, sending a full tablet report...
I20260811 09:19:30.645730 18827 tablet_bootstrap.cc:492] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: Bootstrap starting.
I20260811 09:19:30.648762 18827 tablet_bootstrap.cc:654] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:30.649189 18828 tablet_bootstrap.cc:492] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: Bootstrap starting.
I20260811 09:19:30.650254 18827 log.cc:824] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:30.650611 18829 tablet_bootstrap.cc:492] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d: Bootstrap starting.
I20260811 09:19:30.652422 18828 tablet_bootstrap.cc:654] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:30.653532 18829 tablet_bootstrap.cc:654] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:30.653647 18828 log.cc:824] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:30.654003 18827 tablet_bootstrap.cc:492] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: No bootstrap required, opened a new log
I20260811 09:19:30.654312 18827 ts_tablet_manager.cc:1397] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260811 09:19:30.654484 18829 log.cc:824] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:30.657128 18829 tablet_bootstrap.cc:492] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d: No bootstrap required, opened a new log
I20260811 09:19:30.657372 18829 ts_tablet_manager.cc:1397] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 09:19:30.657742 18828 tablet_bootstrap.cc:492] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: No bootstrap required, opened a new log
I20260811 09:19:30.657996 18828 ts_tablet_manager.cc:1397] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260811 09:19:30.664187 18829 raft_consensus.cc:348] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6835b23ae1e54c0bb09a87534735b956" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40415 } } peers { permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 } } peers { permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 } }
I20260811 09:19:30.664531 18829 raft_consensus.cc:374] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:30.664623 18829 raft_consensus.cc:729] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f181f28de2d498cb806c848dd50a05d, State: Initialized, Role: FOLLOWER
I20260811 09:19:30.665184 18829 consensus_queue.cc:260] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6835b23ae1e54c0bb09a87534735b956" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40415 } } peers { permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 } } peers { permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 } }
I20260811 09:19:30.665364 18827 raft_consensus.cc:348] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6835b23ae1e54c0bb09a87534735b956" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40415 } } peers { permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 } } peers { permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 } }
I20260811 09:19:30.665793 18827 raft_consensus.cc:374] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:30.665903 18827 raft_consensus.cc:729] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0189bcdf97b14bf3a397e122c779db28, State: Initialized, Role: FOLLOWER
I20260811 09:19:30.666781 18829 ts_tablet_manager.cc:1428] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260811 09:19:30.667874 18827 consensus_queue.cc:260] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6835b23ae1e54c0bb09a87534735b956" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40415 } } peers { permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 } } peers { permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 } }
I20260811 09:19:30.667950 18828 raft_consensus.cc:348] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6835b23ae1e54c0bb09a87534735b956" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40415 } } peers { permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 } } peers { permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 } }
I20260811 09:19:30.668355 18828 raft_consensus.cc:374] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:30.668454 18828 raft_consensus.cc:729] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6835b23ae1e54c0bb09a87534735b956, State: Initialized, Role: FOLLOWER
I20260811 09:19:30.669698 18828 consensus_queue.cc:260] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6835b23ae1e54c0bb09a87534735b956" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40415 } } peers { permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 } } peers { permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 } }
I20260811 09:19:30.670763 18802 heartbeater.cc:503] Master 127.17.16.126:41737 was elected leader, sending a full tablet report...
I20260811 09:19:30.671502 18828 ts_tablet_manager.cc:1428] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260811 09:19:30.673967 18591 heartbeater.cc:503] Master 127.17.16.126:41737 was elected leader, sending a full tablet report...
I20260811 09:19:30.674002 18827 ts_tablet_manager.cc:1428] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: Time spent starting tablet: real 0.020s user 0.010s sys 0.006s
I20260811 09:19:30.723069 18839 raft_consensus.cc:482] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:19:30.723361 18839 raft_consensus.cc:504] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6835b23ae1e54c0bb09a87534735b956" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40415 } } peers { permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 } } peers { permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 } }
I20260811 09:19:30.725164 18839 leader_election.cc:288] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0189bcdf97b14bf3a397e122c779db28 (127.17.16.66:35503), 1f181f28de2d498cb806c848dd50a05d (127.17.16.65:44845)
I20260811 09:19:30.731941 18805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.725415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:44845 (local address 127.17.16.67:32795)
0811 09:19:30.725573 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:30.725593 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:30.725715 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:30.726173 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:30.726178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:30.726211 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:30.726372 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.726380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.727750 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.727756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.729904 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.729921 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.731207 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.731212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.731367 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.731403 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:19:30.731534 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 09:19:30.731656 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":23}
I20260811 09:19:30.731961 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.725589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:32795 (local address 127.17.16.65:44845)
0811 09:19:30.725812 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.725817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.725837 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:19:30.725915 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.725921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.726008 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.726103 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:30.726514 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.727608 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.730090 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.731048 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.731207 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.731277 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.731590 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.731727 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 09:19:30.732771 18841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.725426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.66:35503 (local address 127.17.16.67:57985)
0811 09:19:30.726302 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:30.726315 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 09:19:30.726437 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:19:30.726825 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:19:30.726831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:19:30.726861 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:19:30.727255 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.727268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.728555 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.728561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.730510 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:19:30.730523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.732067 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.732073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:19:30.732242 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.732278 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:19:30.732379 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 09:19:30.732510 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":168,"thread_start_us":361,"threads_started":1}
I20260811 09:19:30.732788 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.725704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.16.67:57985 (local address 127.17.16.66:35503)
0811 09:19:30.725910 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.725916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.726406 (+ 490us) server_negotiation.cc:407] Connection header received
0811 09:19:30.726564 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.726568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.726640 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.726705 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:19:30.727465 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.728361 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.730705 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.731872 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.732037 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.732100 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.732414 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.732550 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:19:30.733052 18296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6531f95d8ac34b82a40fc8fc5b47e2f1" candidate_uuid: "6835b23ae1e54c0bb09a87534735b956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f181f28de2d498cb806c848dd50a05d" is_pre_election: true
I20260811 09:19:30.733644 18296 raft_consensus.cc:2454] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6835b23ae1e54c0bb09a87534735b956 in term 0.
I20260811 09:19:30.733880 18508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6531f95d8ac34b82a40fc8fc5b47e2f1" candidate_uuid: "6835b23ae1e54c0bb09a87534735b956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0189bcdf97b14bf3a397e122c779db28" is_pre_election: true
I20260811 09:19:30.734285 18621 leader_election.cc:302] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f181f28de2d498cb806c848dd50a05d, 6835b23ae1e54c0bb09a87534735b956; no voters:
I20260811 09:19:30.734436 18508 raft_consensus.cc:2454] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6835b23ae1e54c0bb09a87534735b956 in term 0.
I20260811 09:19:30.734644 18839 raft_consensus.cc:2781] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:19:30.734764 18839 raft_consensus.cc:482] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:19:30.734859 18839 raft_consensus.cc:3037] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:30.737057 18839 raft_consensus.cc:504] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6835b23ae1e54c0bb09a87534735b956" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40415 } } peers { permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 } } peers { permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 } }
I20260811 09:19:30.737624 18839 leader_election.cc:288] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [CANDIDATE]: Term 1 election: Requested vote from peers 0189bcdf97b14bf3a397e122c779db28 (127.17.16.66:35503), 1f181f28de2d498cb806c848dd50a05d (127.17.16.65:44845)
I20260811 09:19:30.737986 18508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6531f95d8ac34b82a40fc8fc5b47e2f1" candidate_uuid: "6835b23ae1e54c0bb09a87534735b956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0189bcdf97b14bf3a397e122c779db28"
I20260811 09:19:30.738083 18296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6531f95d8ac34b82a40fc8fc5b47e2f1" candidate_uuid: "6835b23ae1e54c0bb09a87534735b956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f181f28de2d498cb806c848dd50a05d"
I20260811 09:19:30.738168 18508 raft_consensus.cc:3037] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:30.738266 18296 raft_consensus.cc:3037] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:30.740377 18296 raft_consensus.cc:2454] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6835b23ae1e54c0bb09a87534735b956 in term 1.
I20260811 09:19:30.740388 18508 raft_consensus.cc:2454] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6835b23ae1e54c0bb09a87534735b956 in term 1.
I20260811 09:19:30.740725 18621 leader_election.cc:302] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f181f28de2d498cb806c848dd50a05d, 6835b23ae1e54c0bb09a87534735b956; no voters:
I20260811 09:19:30.741046 18839 raft_consensus.cc:2781] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:30.741475 18839 raft_consensus.cc:686] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 1 LEADER]: Becoming Leader. State: Replica: 6835b23ae1e54c0bb09a87534735b956, State: Running, Role: LEADER
I20260811 09:19:30.741914 18839 consensus_queue.cc:237] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6835b23ae1e54c0bb09a87534735b956" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40415 } } peers { permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 } } peers { permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 } }
I20260811 09:19:30.746335 18065 catalog_manager.cc:5495] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6835b23ae1e54c0bb09a87534735b956 (127.17.16.67). New cstate: current_term: 1 leader_uuid: "6835b23ae1e54c0bb09a87534735b956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6835b23ae1e54c0bb09a87534735b956" member_type: VOTER last_known_addr { host: "127.17.16.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 } health_report { overall_health: UNKNOWN } } }
I20260811 09:19:30.770020 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.764837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43550 (local address 127.17.16.67:40415)
0811 09:19:30.765073 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.765079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.765098 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:19:30.765284 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.765287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.765375 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.765516 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:19:30.765808 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.766583 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.768006 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.768418 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.768524 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.768949 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:19:30.769527 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:19:30.769636 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.769709 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.769774 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 09:19:30.777686 18804 tablet.cc:2369] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:19:30.777904 18296 raft_consensus.cc:1261] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d [term 1 FOLLOWER]: Refusing update from remote peer 6835b23ae1e54c0bb09a87534735b956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:30.778083 18508 raft_consensus.cc:1261] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28 [term 1 FOLLOWER]: Refusing update from remote peer 6835b23ae1e54c0bb09a87534735b956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:19:30.778884 18842 consensus_queue.cc:1035] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f181f28de2d498cb806c848dd50a05d" member_type: VOTER last_known_addr { host: "127.17.16.65" port: 44845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:30.779232 18839 consensus_queue.cc:1035] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0189bcdf97b14bf3a397e122c779db28" member_type: VOTER last_known_addr { host: "127.17.16.66" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:19:30.789914 18846 mvcc.cc:205] Tried to move back new op lower bound from 7317258120296812544 to 7317258120162672640. Current Snapshot: MvccSnapshot[applied={T|T < 7317258120296812544}]
I20260811 09:19:30.791671 18850 mvcc.cc:205] Tried to move back new op lower bound from 7317258120296812544 to 7317258120162672640. Current Snapshot: MvccSnapshot[applied={T|T < 7317258120296812544}]
I20260811 09:19:30.803864 18065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:30.804163 18065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:19:30.807971 18065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2a200eb1df64e4594d5e3c2bb0accca: Sending DeleteTablet for 3 replicas of tablet 6531f95d8ac34b82a40fc8fc5b47e2f1
I20260811 09:19:30.809204 18686 tablet_service.cc:1514] Processing DeleteTablet for tablet 6531f95d8ac34b82a40fc8fc5b47e2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:30 UTC) from {username='slave'} at 127.0.0.1:43534
I20260811 09:19:30.809248 18475 tablet_service.cc:1514] Processing DeleteTablet for tablet 6531f95d8ac34b82a40fc8fc5b47e2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:30 UTC) from {username='slave'} at 127.0.0.1:54978
I20260811 09:19:30.809659 18269 tablet_service.cc:1514] Processing DeleteTablet for tablet 6531f95d8ac34b82a40fc8fc5b47e2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:19:30 UTC) from {username='slave'} at 127.0.0.1:44244
I20260811 09:19:30.810936 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 18161
I20260811 09:19:30.828474 18868 tablet_replica.cc:330] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d: stopping tablet replica
I20260811 09:19:30.829106 18868 raft_consensus.cc:2229] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:30.829771 18868 raft_consensus.cc:2258] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:30.834913 18868 ts_tablet_manager.cc:1905] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 1f181f28de2d498cb806c848dd50a05d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:30.836808 18866 tablet_replica.cc:330] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: stopping tablet replica
I20260811 09:19:30.837301 18866 raft_consensus.cc:2229] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:19:30.838022 18866 raft_consensus.cc:2258] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:30.838587 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.812062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44246 (local address 127.17.16.65:44845)
0811 09:19:30.812243 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.812249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.828748 (+ 16499us) server_negotiation.cc:407] Connection header received
0811 09:19:30.829117 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.829124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.829243 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.829330 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:30.830105 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.831316 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.833222 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.833897 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.833996 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.836849 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:30.836860 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:30.836877 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:30.837666 (+ 789us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:30.837737 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:30.837763 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:30.837851 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.838111 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.838227 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 09:19:30.838595 18867 tablet_replica.cc:330] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: stopping tablet replica
I20260811 09:19:30.838972 18867 raft_consensus.cc:2229] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:19:30.839471 18867 raft_consensus.cc:2258] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:19:30.839702 18364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:30.841758 18867 ts_tablet_manager.cc:1905] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:30.844406 18866 ts_tablet_manager.cc:1905] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:19:30.851580 18867 ts_tablet_manager.cc:1918] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:30.851758 18867 log.cc:1196] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-1/wal/wals/6531f95d8ac34b82a40fc8fc5b47e2f1
I20260811 09:19:30.852384 18867 ts_tablet_manager.cc:1939] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 0189bcdf97b14bf3a397e122c779db28: Deleting consensus metadata
I20260811 09:19:30.854421 18038 catalog_manager.cc:4841] TS 0189bcdf97b14bf3a397e122c779db28 (127.17.16.66:35503): tablet 6531f95d8ac34b82a40fc8fc5b47e2f1 (table test-table [id=df7d475bde0941f2b156d0dd8d2ed75c]) successfully deleted
I20260811 09:19:30.855523 18866 ts_tablet_manager.cc:1918] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:19:30.855716 18866 log.cc:1196] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: Deleting WAL directory at /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439649204149-17473-0/minicluster-data/ts-2/wal/wals/6531f95d8ac34b82a40fc8fc5b47e2f1
I20260811 09:19:30.856249 18866 ts_tablet_manager.cc:1939] T 6531f95d8ac34b82a40fc8fc5b47e2f1 P 6835b23ae1e54c0bb09a87534735b956: Deleting consensus metadata
I20260811 09:19:30.858048 18040 catalog_manager.cc:4841] TS 6835b23ae1e54c0bb09a87534735b956 (127.17.16.67:40415): tablet 6531f95d8ac34b82a40fc8fc5b47e2f1 (table test-table [id=df7d475bde0941f2b156d0dd8d2ed75c]) successfully deleted
I20260811 09:19:30.932715 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 18161
W20260811 09:19:30.953969 18039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.16.65:44845 (error 108)
I20260811 09:19:30.954732 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 18381
I20260811 09:19:30.956125 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.955665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:44845 (local address 127.0.0.1:44256)
0811 09:19:30.955839 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:30.955951 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 09:19:30.956399 18039 catalog_manager.cc:4584] TS 1f181f28de2d498cb806c848dd50a05d (127.17.16.65:44845): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6531f95d8ac34b82a40fc8fc5b47e2f1: Network error: Client connection negotiation failed: client connection to 127.17.16.65:44845: connect: Connection refused (error 111)
I20260811 09:19:30.963137 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:30.955557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54986 (local address 127.17.16.66:35503)
0811 09:19:30.955745 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:30.955752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:30.956035 (+ 283us) server_negotiation.cc:407] Connection header received
0811 09:19:30.956237 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:30.956243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:30.956386 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:30.956506 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:30.957089 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.957882 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.959739 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:30.960120 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:30.960208 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:30.961349 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:30.961358 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:30.961372 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:30.961980 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:30.962054 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:30.962081 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:30.962206 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:30.962469 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 09:19:30.962661 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:19:30.963851 18583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:31.010270 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:31.009602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:44845 (local address 127.0.0.1:44266)
0811 09:19:31.009893 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:31.010032 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 09:19:31.011528 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:31.011093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:44845 (local address 127.0.0.1:44270)
0811 09:19:31.011274 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:31.011362 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 09:19:31.052068 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 18381
I20260811 09:19:31.072198 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:31.071556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:44845 (local address 127.0.0.1:44284)
0811 09:19:31.071890 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:31.071999 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260811 09:19:31.073504 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:31.073062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:44845 (local address 127.0.0.1:44290)
0811 09:19:31.073267 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:31.073352 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 09:19:31.077719 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 18596
I20260811 09:19:31.086895 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:31.078541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.17.16.67:40415)
0811 09:19:31.078730 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:31.078737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:31.078826 (+ 89us) server_negotiation.cc:407] Connection header received
0811 09:19:31.079009 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:31.079014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:31.079147 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:31.079263 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:31.079754 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:31.080990 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:31.082890 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:31.083561 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:31.083711 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:31.085163 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:31.085179 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:31.085199 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:31.086027 (+ 828us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:31.086109 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:31.086136 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:31.086247 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:31.086408 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 09:19:31.086519 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:19:31.087666 18796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:31.154183 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:31.153706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:44845 (local address 127.0.0.1:44300)
0811 09:19:31.153926 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:31.154019 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 09:19:31.155233 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:31.154828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:44845 (local address 127.0.0.1:44302)
0811 09:19:31.155021 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:31.155102 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 09:19:31.208511 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 18596
I20260811 09:19:31.234871 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 18009
I20260811 09:19:31.235996 18112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:31.343410 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:31.342753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.65:44845 (local address 127.0.0.1:44306)
0811 09:19:31.343053 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 09:19:31.343167 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.65:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260811 09:19:31.343397 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 18009
2026-08-11T09:19:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2825 ms)
[----------] 6 tests from EncryptionPolicyTest (22672 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:31 dist-test-slave-ncp9 krb5kdc[18886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:31 dist-test-slave-ncp9 krb5kdc[18886](info): set up 2 sockets
Aug 11 09:19:31 dist-test-slave-ncp9 krb5kdc[18886](info): commencing operation
krb5kdc: starting...
W20260811 09:19:32.592067 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:32 dist-test-slave-ncp9 krb5kdc[18886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:32.919831 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32925
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:33.262157 18909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:33.262506 18909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:33.262655 18909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:33.270046 18909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:33.270187 18909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:33.270262 18909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:33.270344 18909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:33.270390 18909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:33.280534 18909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32925
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:33.281901 18909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:33.283218 18909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:33.295230 18917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:33.295282 18918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:33.295200 18922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:33.296226 18909 server_base.cc:1034] running on GCE node
I20260811 09:19:33.297046 18909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:33.298410 18909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:33.299619 18909 hybrid_clock.cc:648] HybridClock initialized: now 1786439973299571 us; error 30 us; skew 500 ppm
I20260811 09:19:33.299950 18909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:33 dist-test-slave-ncp9 krb5kdc[18886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:33.302977 18909 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:33.304414 18909 webserver.cc:460] Webserver started at http://127.0.0.1:34433/ using document root <none> and password file <none>
I20260811 09:19:33.305128 18909 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:33.305248 18909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:33.305531 18909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:33.308440 18909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "3c863110a1654bd0bfdcc6bd44ef15cf"
format_stamp: "Formatted at 2026-08-11 09:19:33 on dist-test-slave-ncp9"
server_key: "0fab2b91740265c88738a0ef284b7efe"
server_key_iv: "0581da0ad86368f373d86716db3f7aa7"
server_key_version: "encryptionkey@0"
I20260811 09:19:33.309232 18909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "3c863110a1654bd0bfdcc6bd44ef15cf"
format_stamp: "Formatted at 2026-08-11 09:19:33 on dist-test-slave-ncp9"
server_key: "0fab2b91740265c88738a0ef284b7efe"
server_key_iv: "0581da0ad86368f373d86716db3f7aa7"
server_key_version: "encryptionkey@0"
I20260811 09:19:33.314116 18909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 09:19:33.317497 18940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:33.318698 18909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:19:33.318863 18909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "3c863110a1654bd0bfdcc6bd44ef15cf"
format_stamp: "Formatted at 2026-08-11 09:19:33 on dist-test-slave-ncp9"
server_key: "0fab2b91740265c88738a0ef284b7efe"
server_key_iv: "0581da0ad86368f373d86716db3f7aa7"
server_key_version: "encryptionkey@0"
I20260811 09:19:33.319046 18909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:33.333227 18909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:33.336949 18909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:33.337214 18909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:33.354593 18909 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32925
I20260811 09:19:33.354624 19026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32925 every 8 connection(s)
I20260811 09:19:33.355940 18909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:33.359803 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 18909
I20260811 09:19:33.359843 19028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:33.360019 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:33.360847 17473 external_mini_cluster.cc:1428] Setting key 258101bb5e284fe2ad128ac5026154d4
I20260811 09:19:33.373436 19028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf: Bootstrap starting.
Aug 11 09:19:33 dist-test-slave-ncp9 krb5kdc[18886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:19:33.377321 19028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:33.378410 19028 log.cc:824] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:33.381556 19028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf: No bootstrap required, opened a new log
I20260811 09:19:33.385109 19035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:33.362812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58259 (local address 127.0.0.1:32925)
0811 09:19:33.363629 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:33.363640 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:33.363681 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:19:33.365939 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:33.365965 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:33.366350 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:33.366712 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:33.368479 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:33.369860 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:33.371613 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:33.372266 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:33.372393 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:33.377253 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:33.377262 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:33.377276 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:33.377317 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:33.380785 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:33.381675 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:33.381683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:33.381691 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:33.381875 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:33.382344 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:33.382348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:33.382350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:33.382914 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:33.383271 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:33.383734 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 09:19:33.384044 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":381,"threads_started":1}
I20260811 09:19:33.389137 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:33.390003 19028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c863110a1654bd0bfdcc6bd44ef15cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32925 } }
I20260811 09:19:33.390342 19028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:33.390429 19028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c863110a1654bd0bfdcc6bd44ef15cf, State: Initialized, Role: FOLLOWER
I20260811 09:19:33.390967 19028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c863110a1654bd0bfdcc6bd44ef15cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32925 } }
I20260811 09:19:33.391201 19028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:33.391328 19028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:33.391481 19028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:33.394095 19028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c863110a1654bd0bfdcc6bd44ef15cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32925 } }
I20260811 09:19:33.394655 19028 leader_election.cc:302] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c863110a1654bd0bfdcc6bd44ef15cf; no voters:
I20260811 09:19:33.395262 19028 leader_election.cc:288] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:33.395401 19044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:33.396031 19044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [term 1 LEADER]: Becoming Leader. State: Replica: 3c863110a1654bd0bfdcc6bd44ef15cf, State: Running, Role: LEADER
I20260811 09:19:33.396629 19044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c863110a1654bd0bfdcc6bd44ef15cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32925 } }
I20260811 09:19:33.398653 19028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:33.406751 19045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c863110a1654bd0bfdcc6bd44ef15cf. Latest consensus state: current_term: 1 leader_uuid: "3c863110a1654bd0bfdcc6bd44ef15cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c863110a1654bd0bfdcc6bd44ef15cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32925 } } }
I20260811 09:19:33.407146 19045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:33.407575 19046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c863110a1654bd0bfdcc6bd44ef15cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c863110a1654bd0bfdcc6bd44ef15cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32925 } } }
I20260811 09:19:33.407887 19046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:33.410152 19059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:33.412227 19035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:33.393889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41715 (local address 127.0.0.1:32925)
0811 09:19:33.394071 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:33.394077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:33.394439 (+ 362us) server_negotiation.cc:407] Connection header received
0811 09:19:33.394568 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:33.394573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:33.394653 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:33.394757 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:33.396516 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:33.397744 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:33.399683 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:33.400485 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:33.400607 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:33.404168 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:33.404175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:33.404181 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:33.404222 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:33.409339 (+ 5117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:33.410260 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:33.410264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:33.410267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:33.410404 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:33.410885 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:33.410888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:33.410890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:33.411193 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:33.411390 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:33.411814 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 09:19:33.411949 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:19:33.415987 19059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:33.425232 19059 catalog_manager.cc:1347] Generated new cluster ID: b1468dc2a31b481aab30db765c0e4b37
I20260811 09:19:33.425338 19059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:33.431830 19059 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:33.433079 19059 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:33.442528 19059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c863110a1654bd0bfdcc6bd44ef15cf: Generated new TSK 0
I20260811 09:19:33.443595 19059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:33.455286 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 18909
I20260811 09:19:33.457086 19021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:33.545409 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 18909
W20260811 09:19:33.570957 18882 connection.cc:535] client connection to 127.0.0.1:32925 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32925): Connection reset by peer (error 104)
2026-08-11T09:19:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2197 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:33 dist-test-slave-ncp9 krb5kdc[19078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:33 dist-test-slave-ncp9 krb5kdc[19078](info): set up 2 sockets
Aug 11 09:19:33 dist-test-slave-ncp9 krb5kdc[19078](info): commencing operation
krb5kdc: starting...
W20260811 09:19:34.803539 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:35 dist-test-slave-ncp9 krb5kdc[19078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:35.130340 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36487
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:35.480818 19099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:35.481153 19099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:35.481258 19099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:35.488382 19099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:35.488533 19099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:35.488592 19099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:35.488653 19099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:35.488698 19099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:35.498667 19099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36487
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:35.499933 19099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:35.501262 19099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:35.512626 19111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:35.512604 19114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:35.512583 19108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:35.513695 19099 server_base.cc:1034] running on GCE node
I20260811 09:19:35.514433 19099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:35.515802 19099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:35.516981 19099 hybrid_clock.cc:648] HybridClock initialized: now 1786439975516934 us; error 34 us; skew 500 ppm
I20260811 09:19:35.517314 19099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:35 dist-test-slave-ncp9 krb5kdc[19078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439975, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:35.528436 19099 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:35.531237 19099 webserver.cc:460] Webserver started at http://127.0.0.1:40107/ using document root <none> and password file <none>
I20260811 09:19:35.531916 19099 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:35.532029 19099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:35.532339 19099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:35.535435 19099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "3c7d1d923a00404694edcf6e08cb1765"
format_stamp: "Formatted at 2026-08-11 09:19:35 on dist-test-slave-ncp9"
server_key: "c5736fd8cc7ab1e43f6702e273c69301"
server_key_iv: "1f44d43368291b6e592b9d99c9af93a0"
server_key_version: "encryptionkey@0"
I20260811 09:19:35.536197 19099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "3c7d1d923a00404694edcf6e08cb1765"
format_stamp: "Formatted at 2026-08-11 09:19:35 on dist-test-slave-ncp9"
server_key: "c5736fd8cc7ab1e43f6702e273c69301"
server_key_iv: "1f44d43368291b6e592b9d99c9af93a0"
server_key_version: "encryptionkey@0"
I20260811 09:19:35.541474 19099 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 09:19:35.544943 19128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:35.546187 19099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:19:35.546347 19099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "3c7d1d923a00404694edcf6e08cb1765"
format_stamp: "Formatted at 2026-08-11 09:19:35 on dist-test-slave-ncp9"
server_key: "c5736fd8cc7ab1e43f6702e273c69301"
server_key_iv: "1f44d43368291b6e592b9d99c9af93a0"
server_key_version: "encryptionkey@0"
I20260811 09:19:35.546520 19099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:35.571602 19099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:35.576764 19099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:35.577173 19099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:35.596262 19099 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36487
I20260811 09:19:35.596282 19214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36487 every 8 connection(s)
I20260811 09:19:35.597663 19099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:35.601390 19216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:35.601842 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 19099
I20260811 09:19:35.602053 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:35.602632 17473 external_mini_cluster.cc:1428] Setting key ef5945f2e6509bce154d28c859ecb92b
I20260811 09:19:35.614884 19216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765: Bootstrap starting.
I20260811 09:19:35.616465 19221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:35.604438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36587 (local address 127.0.0.1:36487)
0811 09:19:35.605270 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:35.605282 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:35.605327 (+ 45us) server_negotiation.cc:407] Connection header received
0811 09:19:35.607551 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:35.607579 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:35.607998 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:35.608338 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:35.608954 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:35.610246 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:35.612144 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:35.612834 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:35.613010 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:35.614219 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:35.614227 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:35.614242 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:35.614760 (+ 518us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:35.614835 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:35.614869 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:35.614980 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:35.615311 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 09:19:35.615494 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":437,"threads_started":1}
I20260811 09:19:35.618223 19216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:35.619447 19216 log.cc:824] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:35.620580 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:35.622575 19216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765: No bootstrap required, opened a new log
I20260811 09:19:35.632122 19221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:35.625656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47815 (local address 127.0.0.1:36487)
0811 09:19:35.625856 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:35.625861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:35.626204 (+ 343us) server_negotiation.cc:407] Connection header received
0811 09:19:35.626371 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:35.626375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:35.626435 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:35.626516 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:35.627063 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:35.627823 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:35.629668 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:35.630229 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:35.630328 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:35.631433 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:35.631438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:35.631441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:35.631458 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:35.631506 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:35.631521 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:35.631601 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:35.631788 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 09:19:35.631829 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:19:35.632616 19216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7d1d923a00404694edcf6e08cb1765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } }
I20260811 09:19:35.633112 19216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:35.633222 19216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7d1d923a00404694edcf6e08cb1765, State: Initialized, Role: FOLLOWER
I20260811 09:19:35.633818 19216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7d1d923a00404694edcf6e08cb1765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } }
I20260811 09:19:35.634032 19216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:35.634135 19216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:35.634258 19216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:35.636343 19216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7d1d923a00404694edcf6e08cb1765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } }
I20260811 09:19:35.636982 19216 leader_election.cc:302] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c7d1d923a00404694edcf6e08cb1765; no voters:
I20260811 09:19:35.637493 19216 leader_election.cc:288] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:35.637622 19232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:35.638259 19232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [term 1 LEADER]: Becoming Leader. State: Replica: 3c7d1d923a00404694edcf6e08cb1765, State: Running, Role: LEADER
I20260811 09:19:35.638744 19232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7d1d923a00404694edcf6e08cb1765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } }
I20260811 09:19:35.639420 19216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:35.641567 19234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c7d1d923a00404694edcf6e08cb1765. Latest consensus state: current_term: 1 leader_uuid: "3c7d1d923a00404694edcf6e08cb1765" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7d1d923a00404694edcf6e08cb1765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } } }
I20260811 09:19:35.641743 19233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c7d1d923a00404694edcf6e08cb1765" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7d1d923a00404694edcf6e08cb1765" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36487 } } }
I20260811 09:19:35.641978 19234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:35.642027 19233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:35.643611 19248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:35.648070 19248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:35.656234 19248 catalog_manager.cc:1347] Generated new cluster ID: ea00a92b2f394a5399df943c60a8a497
I20260811 09:19:35.656347 19248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:35.693996 19248 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:35.695339 19248 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:35.702924 19248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c7d1d923a00404694edcf6e08cb1765: Generated new TSK 0
I20260811 09:19:35.703786 19248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:35.718250 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19099
W20260811 09:19:35.719427 19210 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36587
W20260811 09:19:35.720384 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19099: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:19:35.720633 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 19099
W20260811 09:19:35.747398 19071 connection.cc:535] client connection to 127.0.0.1:36487 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36487): Connection reset by peer (error 104)
2026-08-11T09:19:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2176 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:35 dist-test-slave-ncp9 krb5kdc[19275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:35 dist-test-slave-ncp9 krb5kdc[19275](info): set up 2 sockets
Aug 11 09:19:35 dist-test-slave-ncp9 krb5kdc[19275](info): commencing operation
krb5kdc: starting...
W20260811 09:19:36.964365 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:37 dist-test-slave-ncp9 krb5kdc[19275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:37.282248 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:37.618752 19299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:37.619028 19299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:37.619154 19299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:37.626398 19299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:37.626518 19299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:37.626595 19299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:37.626672 19299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:37.626731 19299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:37.636574 19299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:37.637846 19299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:37.639101 19299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:37.650413 19306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:37.650365 19310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:37.650440 19307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:37.651438 19299 server_base.cc:1034] running on GCE node
I20260811 09:19:37.652195 19299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:37.653528 19299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:37.654721 19299 hybrid_clock.cc:648] HybridClock initialized: now 1786439977654664 us; error 48 us; skew 500 ppm
I20260811 09:19:37.655043 19299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:37 dist-test-slave-ncp9 krb5kdc[19275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439977, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:37.658126 19299 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:37.659471 19299 webserver.cc:460] Webserver started at http://127.0.0.1:38899/ using document root <none> and password file <none>
I20260811 09:19:37.660110 19299 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:37.660239 19299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:37.660513 19299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:37.663482 19299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "ad73a73215d849b691fbd82db60b568b"
format_stamp: "Formatted at 2026-08-11 09:19:37 on dist-test-slave-ncp9"
server_key: "6a0878f2b18c65b550956f7c3f96802c"
server_key_iv: "e13b86dd62f9c9233557a9638987134b"
server_key_version: "encryptionkey@0"
I20260811 09:19:37.664234 19299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "ad73a73215d849b691fbd82db60b568b"
format_stamp: "Formatted at 2026-08-11 09:19:37 on dist-test-slave-ncp9"
server_key: "6a0878f2b18c65b550956f7c3f96802c"
server_key_iv: "e13b86dd62f9c9233557a9638987134b"
server_key_version: "encryptionkey@0"
I20260811 09:19:37.669171 19299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 09:19:37.672449 19327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:37.673743 19299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:19:37.673900 19299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "ad73a73215d849b691fbd82db60b568b"
format_stamp: "Formatted at 2026-08-11 09:19:37 on dist-test-slave-ncp9"
server_key: "6a0878f2b18c65b550956f7c3f96802c"
server_key_iv: "e13b86dd62f9c9233557a9638987134b"
server_key_version: "encryptionkey@0"
I20260811 09:19:37.674073 19299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:37.679533 19299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:37.683590 19299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:37.683861 19299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:37.701249 19299 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41419
I20260811 09:19:37.701297 19420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41419 every 8 connection(s)
I20260811 09:19:37.702574 19299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:37.706135 19422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:37.712797 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 19299
I20260811 09:19:37.712982 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:37.713403 17473 external_mini_cluster.cc:1428] Setting key 402252d89ba64f9f7abf455615bcaa06
I20260811 09:19:37.716158 19422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b: Bootstrap starting.
I20260811 09:19:37.719312 19422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:37.720431 19422 log.cc:824] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:37.720827 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:37.715063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35925 (local address 127.0.0.1:41419)
0811 09:19:37.715824 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:37.715835 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:37.715877 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:19:37.717604 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:37.717629 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:37.717955 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:37.718288 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:37.719275 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:37.719282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:37.719293 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:37.719312 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:37.719379 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:37.719407 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:37.719602 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:37.719756 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 09:19:37.719941 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":329,"threads_started":1}
I20260811 09:19:37.723158 19422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b: No bootstrap required, opened a new log
I20260811 09:19:37.724277 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:37.730602 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:37.728402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36437 (local address 127.0.0.1:41419)
0811 09:19:37.728557 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:37.728563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:37.728871 (+ 308us) server_negotiation.cc:407] Connection header received
0811 09:19:37.729044 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:37.729052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:37.729110 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:37.729212 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:37.730035 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:37.730045 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:37.730050 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:37.730069 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:37.730114 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:37.730137 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:37.730258 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:37.730300 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 09:19:37.730349 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:19:37.731110 19422 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad73a73215d849b691fbd82db60b568b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } }
I20260811 09:19:37.731557 19422 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:37.731668 19422 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad73a73215d849b691fbd82db60b568b, State: Initialized, Role: FOLLOWER
I20260811 09:19:37.732213 19422 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad73a73215d849b691fbd82db60b568b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } }
I20260811 09:19:37.732432 19422 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:37.732537 19422 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:37.732666 19422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:37.734858 19422 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad73a73215d849b691fbd82db60b568b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } }
I20260811 09:19:37.735289 19422 leader_election.cc:302] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad73a73215d849b691fbd82db60b568b; no voters:
I20260811 09:19:37.735807 19422 leader_election.cc:288] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:37.735927 19441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:37.736971 19441 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [term 1 LEADER]: Becoming Leader. State: Replica: ad73a73215d849b691fbd82db60b568b, State: Running, Role: LEADER
I20260811 09:19:37.737564 19441 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad73a73215d849b691fbd82db60b568b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } }
I20260811 09:19:37.737814 19422 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:37.740316 19442 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad73a73215d849b691fbd82db60b568b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad73a73215d849b691fbd82db60b568b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } } }
I20260811 09:19:37.740485 19444 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad73a73215d849b691fbd82db60b568b. Latest consensus state: current_term: 1 leader_uuid: "ad73a73215d849b691fbd82db60b568b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad73a73215d849b691fbd82db60b568b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41419 } } }
I20260811 09:19:37.740737 19444 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:37.740716 19442 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:37.742185 19453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:37.747249 19453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:37.756423 19453 catalog_manager.cc:1347] Generated new cluster ID: d6d4c332bb9542b19e21154edc6ebdd5
I20260811 09:19:37.756542 19453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:37.774202 19453 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:37.774617 19453 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:37.786155 19453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad73a73215d849b691fbd82db60b568b: Generated new TSK 0
I20260811 09:19:37.786901 19453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:37.803375 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19299
W20260811 09:19:37.804395 19416 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35925
W20260811 09:19:37.805101 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19299: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:19:37.805228 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 19299
2026-08-11T09:19:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2079 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:37 dist-test-slave-ncp9 krb5kdc[19469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:37 dist-test-slave-ncp9 krb5kdc[19469](info): set up 2 sockets
Aug 11 09:19:37 dist-test-slave-ncp9 krb5kdc[19469](info): commencing operation
krb5kdc: starting...
W20260811 09:19:39.065459 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:39 dist-test-slave-ncp9 krb5kdc[19469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:39.392127 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36035
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:39.746484 19492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:39.746820 19492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:39.746979 19492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:39.754302 19492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:39.754434 19492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:39.754534 19492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:39.754631 19492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:39.754701 19492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:39.764707 19492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:39.766086 19492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:39.767432 19492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:39.779222 19502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:39.779254 19506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:39.779275 19514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:39.780267 19492 server_base.cc:1034] running on GCE node
I20260811 09:19:39.781096 19492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:39.782497 19492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:39.783703 19492 hybrid_clock.cc:648] HybridClock initialized: now 1786439979783657 us; error 32 us; skew 500 ppm
I20260811 09:19:39.784068 19492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:39 dist-test-slave-ncp9 krb5kdc[19469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:39.787379 19492 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:39.789043 19492 webserver.cc:460] Webserver started at http://127.0.0.1:42911/ using document root <none> and password file <none>
I20260811 09:19:39.789736 19492 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:39.789871 19492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:39.790184 19492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:39.793401 19492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "baacf9c0d63e4bd688e4b35d37189549"
format_stamp: "Formatted at 2026-08-11 09:19:39 on dist-test-slave-ncp9"
server_key: "d8ff698f36bfd430983a6edf1ad7f0d0"
server_key_iv: "293751b6c3a7c9ed03f46f6864a57594"
server_key_version: "encryptionkey@0"
I20260811 09:19:39.794258 19492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "baacf9c0d63e4bd688e4b35d37189549"
format_stamp: "Formatted at 2026-08-11 09:19:39 on dist-test-slave-ncp9"
server_key: "d8ff698f36bfd430983a6edf1ad7f0d0"
server_key_iv: "293751b6c3a7c9ed03f46f6864a57594"
server_key_version: "encryptionkey@0"
I20260811 09:19:39.799681 19492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:19:39.803347 19530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:39.804656 19492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 09:19:39.804829 19492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "baacf9c0d63e4bd688e4b35d37189549"
format_stamp: "Formatted at 2026-08-11 09:19:39 on dist-test-slave-ncp9"
server_key: "d8ff698f36bfd430983a6edf1ad7f0d0"
server_key_iv: "293751b6c3a7c9ed03f46f6864a57594"
server_key_version: "encryptionkey@0"
I20260811 09:19:39.805024 19492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:39.830622 19492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:39.834510 19492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:39.834781 19492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:39.852227 19492 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36035
I20260811 09:19:39.852262 19622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36035 every 8 connection(s)
I20260811 09:19:39.853624 19492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:39.854084 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 19492
I20260811 09:19:39.854255 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:39.854815 17473 external_mini_cluster.cc:1428] Setting key f2d543a51c95fe1ab21044f530fddafa
I20260811 09:19:39.857992 19623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:19:39 dist-test-slave-ncp9 krb5kdc[19469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:19:39.868710 19623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549: Bootstrap starting.
I20260811 09:19:39.872027 19623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:39.873350 19623 log.cc:824] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:39.876515 19623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549: No bootstrap required, opened a new log
I20260811 09:19:39.876914 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:39.856645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47543 (local address 127.0.0.1:36035)
0811 09:19:39.857441 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:39.857453 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:39.857527 (+ 74us) server_negotiation.cc:407] Connection header received
0811 09:19:39.859204 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:39.859228 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:39.859612 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:39.859973 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:39.861513 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:39.862908 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:39.864175 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:39.864917 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:39.865094 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:39.869397 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:39.869406 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:39.869421 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:39.869462 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:39.872488 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:39.873354 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:39.873361 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:39.873367 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:39.873590 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:39.874062 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:39.874067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:39.874070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:39.874823 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:39.875038 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:39.875497 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 09:19:39.875848 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":360,"threads_started":1}
I20260811 09:19:39.880959 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:39.886541 19623 raft_consensus.cc:348] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baacf9c0d63e4bd688e4b35d37189549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36035 } }
I20260811 09:19:39.886922 19623 raft_consensus.cc:374] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:39.887040 19623 raft_consensus.cc:729] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baacf9c0d63e4bd688e4b35d37189549, State: Initialized, Role: FOLLOWER
I20260811 09:19:39.887663 19623 consensus_queue.cc:260] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baacf9c0d63e4bd688e4b35d37189549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36035 } }
I20260811 09:19:39.887879 19623 raft_consensus.cc:388] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:39.888012 19623 raft_consensus.cc:482] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:39.888170 19623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:39.890938 19623 raft_consensus.cc:504] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baacf9c0d63e4bd688e4b35d37189549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36035 } }
I20260811 09:19:39.891444 19623 leader_election.cc:302] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: baacf9c0d63e4bd688e4b35d37189549; no voters:
I20260811 09:19:39.892060 19623 leader_election.cc:288] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:39.892172 19638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:39.893177 19638 raft_consensus.cc:686] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [term 1 LEADER]: Becoming Leader. State: Replica: baacf9c0d63e4bd688e4b35d37189549, State: Running, Role: LEADER
I20260811 09:19:39.893873 19638 consensus_queue.cc:237] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baacf9c0d63e4bd688e4b35d37189549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36035 } }
I20260811 09:19:39.896238 19623 sys_catalog.cc:564] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:39.896811 19639 sys_catalog.cc:455] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [sys.catalog]: SysCatalogTable state changed. Reason: New leader baacf9c0d63e4bd688e4b35d37189549. Latest consensus state: current_term: 1 leader_uuid: "baacf9c0d63e4bd688e4b35d37189549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baacf9c0d63e4bd688e4b35d37189549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36035 } } }
I20260811 09:19:39.897336 19639 sys_catalog.cc:458] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:39.898293 19643 sys_catalog.cc:455] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "baacf9c0d63e4bd688e4b35d37189549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baacf9c0d63e4bd688e4b35d37189549" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36035 } } }
I20260811 09:19:39.898666 19643 sys_catalog.cc:458] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:39.901249 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:39.886231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51321 (local address 127.0.0.1:36035)
0811 09:19:39.886411 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:39.886417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:39.886866 (+ 449us) server_negotiation.cc:407] Connection header received
0811 09:19:39.887048 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:39.887054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:39.887136 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:39.887242 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:39.888846 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:39.890000 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:39.891187 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:39.891699 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:39.891810 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:39.894437 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:39.894446 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:39.894451 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:39.894491 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:39.898135 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:39.899046 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:39.899052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:39.899055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:39.899285 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:39.899661 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:39.899667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:39.899671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:39.899972 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:39.900247 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:39.900556 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 09:19:39.900760 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:19:39.904656 19648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:39.909533 19648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:39.917752 19648 catalog_manager.cc:1347] Generated new cluster ID: 237bfa9c49b74f25a6a5245182f619fa
I20260811 09:19:39.917877 19648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:39.946472 19648 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:39.947815 19648 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:39.956133 19648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P baacf9c0d63e4bd688e4b35d37189549: Generated new TSK 0
I20260811 09:19:39.957221 19648 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:39.974877 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19492
I20260811 09:19:39.975850 19611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:40.070228 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 19492
2026-08-11T09:19:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2266 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:40 dist-test-slave-ncp9 krb5kdc[19677](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:40 dist-test-slave-ncp9 krb5kdc[19677](info): set up 2 sockets
Aug 11 09:19:40 dist-test-slave-ncp9 krb5kdc[19677](info): commencing operation
krb5kdc: starting...
W20260811 09:19:41.319904 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:41 dist-test-slave-ncp9 krb5kdc[19677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:41.642395 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:41.990130 19708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:41.990420 19708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:41.990525 19708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:41.997783 19708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:41.997892 19708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:41.997949 19708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:41.998032 19708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:41.998088 19708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:42.008931 19708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:42.010239 19708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:42.011787 19708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:42.026189 19714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:42.026422 19716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:42.026613 19713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:42.030275 19708 server_base.cc:1034] running on GCE node
I20260811 09:19:42.031234 19708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:42.033118 19708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:42.034361 19708 hybrid_clock.cc:648] HybridClock initialized: now 1786439982034309 us; error 35 us; skew 500 ppm
I20260811 09:19:42.034788 19708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:42 dist-test-slave-ncp9 krb5kdc[19677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:42.042929 19708 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:42.044871 19708 webserver.cc:460] Webserver started at http://127.0.0.1:42475/ using document root <none> and password file <none>
I20260811 09:19:42.045890 19708 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:42.046054 19708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:42.046435 19708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:42.050911 19708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "bf013615227443e3bc10f18ad246ce92"
format_stamp: "Formatted at 2026-08-11 09:19:42 on dist-test-slave-ncp9"
server_key: "75a43154e2ab84c3a06de66b0885175f"
server_key_iv: "1e1923c25dca15bd30d30915f2edf5d2"
server_key_version: "encryptionkey@0"
I20260811 09:19:42.052035 19708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "bf013615227443e3bc10f18ad246ce92"
format_stamp: "Formatted at 2026-08-11 09:19:42 on dist-test-slave-ncp9"
server_key: "75a43154e2ab84c3a06de66b0885175f"
server_key_iv: "1e1923c25dca15bd30d30915f2edf5d2"
server_key_version: "encryptionkey@0"
I20260811 09:19:42.059096 19708 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 09:19:42.063921 19728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:42.065598 19708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:19:42.065796 19708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "bf013615227443e3bc10f18ad246ce92"
format_stamp: "Formatted at 2026-08-11 09:19:42 on dist-test-slave-ncp9"
server_key: "75a43154e2ab84c3a06de66b0885175f"
server_key_iv: "1e1923c25dca15bd30d30915f2edf5d2"
server_key_version: "encryptionkey@0"
I20260811 09:19:42.066047 19708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:42.089532 19708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:42.095023 19708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:42.095348 19708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:42.115195 19815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46161 every 8 connection(s)
I20260811 09:19:42.115190 19708 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46161
I20260811 09:19:42.116688 19708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:42.121627 19816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:42.123869 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 19708
I20260811 09:19:42.124039 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:42.124476 17473 external_mini_cluster.cc:1428] Setting key 5f8e1b7ec881aee98a47cc4122af3d75
I20260811 09:19:42.136241 19816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92: Bootstrap starting.
I20260811 09:19:42.140189 19816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:42.140450 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:42.126152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39965 (local address 127.0.0.1:46161)
0811 09:19:42.126949 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:42.126964 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:42.127014 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:19:42.128945 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:42.128976 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:42.129488 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:42.129886 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:42.131106 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:42.133066 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:42.135032 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:42.136032 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:42.136200 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:42.137689 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:42.137700 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:42.137716 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:42.138442 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:42.138595 (+ 153us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:42.138628 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:42.138743 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:42.138956 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 09:19:42.139213 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":561,"thread_start_us":329,"threads_started":1}
I20260811 09:19:42.141561 19816 log.cc:824] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:42.144440 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:42.144716 19816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92: No bootstrap required, opened a new log
I20260811 09:19:42.153353 19816 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf013615227443e3bc10f18ad246ce92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260811 09:19:42.153715 19816 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:42.153826 19816 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf013615227443e3bc10f18ad246ce92, State: Initialized, Role: FOLLOWER
I20260811 09:19:42.154418 19816 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf013615227443e3bc10f18ad246ce92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260811 09:19:42.154709 19816 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:42.154860 19816 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:42.155054 19816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:42.157298 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:42.149837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32835 (local address 127.0.0.1:46161)
0811 09:19:42.150009 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:42.150015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:42.150625 (+ 610us) server_negotiation.cc:407] Connection header received
0811 09:19:42.150820 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:42.150826 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:42.150899 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:42.151021 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:42.151596 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:42.152680 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:42.154474 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:42.155168 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:42.155292 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:42.156523 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:42.156530 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:42.156534 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:42.156554 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:42.156601 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:42.156624 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:42.156717 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:42.156860 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 09:19:42.156996 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 09:19:42.158001 19816 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf013615227443e3bc10f18ad246ce92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260811 09:19:42.158543 19816 leader_election.cc:302] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf013615227443e3bc10f18ad246ce92; no voters:
I20260811 09:19:42.159145 19816 leader_election.cc:288] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:42.159260 19829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:42.161381 19816 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:42.161535 19829 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [term 1 LEADER]: Becoming Leader. State: Replica: bf013615227443e3bc10f18ad246ce92, State: Running, Role: LEADER
I20260811 09:19:42.162241 19829 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf013615227443e3bc10f18ad246ce92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260811 09:19:42.165210 19832 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf013615227443e3bc10f18ad246ce92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf013615227443e3bc10f18ad246ce92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } } }
I20260811 09:19:42.165608 19832 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:42.166507 19838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:42.166332 19833 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf013615227443e3bc10f18ad246ce92. Latest consensus state: current_term: 1 leader_uuid: "bf013615227443e3bc10f18ad246ce92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf013615227443e3bc10f18ad246ce92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } } }
I20260811 09:19:42.167017 19833 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:42.178719 19838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:42.197126 19838 catalog_manager.cc:1347] Generated new cluster ID: 96877975e90248e79e056b377685669c
I20260811 09:19:42.197266 19838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:42.216303 19838 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:42.218801 19838 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:42.229460 19838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf013615227443e3bc10f18ad246ce92: Generated new TSK 0
I20260811 09:19:42.230317 19838 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:42.247989 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19708
W20260811 09:19:42.249550 19809 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39965
W20260811 09:19:42.250384 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19708: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:19:42.250551 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 19708
2026-08-11T09:19:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2180 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:42 dist-test-slave-ncp9 krb5kdc[19866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:42 dist-test-slave-ncp9 krb5kdc[19866](info): set up 2 sockets
Aug 11 09:19:42 dist-test-slave-ncp9 krb5kdc[19866](info): commencing operation
krb5kdc: starting...
W20260811 09:19:43.521749 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.008s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:43 dist-test-slave-ncp9 krb5kdc[19866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:43.854259 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:44.200327 19903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:44.200626 19903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:44.200759 19903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:44.207827 19903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:44.207937 19903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:44.208010 19903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:44.208093 19903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:44.208153 19903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:44.217818 19903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:44.219146 19903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:44.220472 19903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:44.232180 19910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:44.232141 19913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:44.232378 19909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:44.232993 19903 server_base.cc:1034] running on GCE node
I20260811 09:19:44.233764 19903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:44.235154 19903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:44.236361 19903 hybrid_clock.cc:648] HybridClock initialized: now 1786439984236321 us; error 29 us; skew 500 ppm
I20260811 09:19:44.236685 19903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:44 dist-test-slave-ncp9 krb5kdc[19866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:44.239699 19903 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:44.241099 19903 webserver.cc:460] Webserver started at http://127.0.0.1:46703/ using document root <none> and password file <none>
I20260811 09:19:44.241745 19903 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:44.241891 19903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:44.242228 19903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:44.245297 19903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "ec3829c88a9e4e93b35b87b8fa28830d"
format_stamp: "Formatted at 2026-08-11 09:19:44 on dist-test-slave-ncp9"
server_key: "294303ef1e6c6abb88a1a0cace7373b9"
server_key_iv: "fef3e6a99df53235ad4a5eb2a6e18b4b"
server_key_version: "encryptionkey@0"
I20260811 09:19:44.246215 19903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "ec3829c88a9e4e93b35b87b8fa28830d"
format_stamp: "Formatted at 2026-08-11 09:19:44 on dist-test-slave-ncp9"
server_key: "294303ef1e6c6abb88a1a0cace7373b9"
server_key_iv: "fef3e6a99df53235ad4a5eb2a6e18b4b"
server_key_version: "encryptionkey@0"
I20260811 09:19:44.251221 19903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:19:44.254705 19925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:44.255959 19903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:19:44.256121 19903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "ec3829c88a9e4e93b35b87b8fa28830d"
format_stamp: "Formatted at 2026-08-11 09:19:44 on dist-test-slave-ncp9"
server_key: "294303ef1e6c6abb88a1a0cace7373b9"
server_key_iv: "fef3e6a99df53235ad4a5eb2a6e18b4b"
server_key_version: "encryptionkey@0"
I20260811 09:19:44.256316 19903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:44.261858 19903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:44.265651 19903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:44.265926 19903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:44.283258 19903 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33637
I20260811 09:19:44.283306 20018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33637 every 8 connection(s)
I20260811 09:19:44.284571 19903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:44.288206 20020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:44.293905 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 19903
I20260811 09:19:44.294071 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:44.294489 17473 external_mini_cluster.cc:1428] Setting key 036929c534464091a28b8ae0e4595993
I20260811 09:19:44.298656 20020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d: Bootstrap starting.
I20260811 09:19:44.302145 20020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:44.302654 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:44.296222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52289 (local address 127.0.0.1:33637)
0811 09:19:44.297073 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:44.297083 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:44.297123 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:19:44.298703 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:44.298729 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:44.299061 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:44.299354 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:44.300425 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:44.300432 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:44.300444 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:44.300457 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:44.300523 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:44.300543 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:44.300722 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:44.301479 (+ 757us) server_negotiation.cc:299] Negotiation successful
0811 09:19:44.301713 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":380,"threads_started":1}
I20260811 09:19:44.303236 20020 log.cc:824] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:44.306010 20020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d: No bootstrap required, opened a new log
I20260811 09:19:44.306037 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:44.312268 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:44.310123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51809 (local address 127.0.0.1:33637)
0811 09:19:44.310290 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:44.310295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:44.310558 (+ 263us) server_negotiation.cc:407] Connection header received
0811 09:19:44.310689 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:44.310695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:44.310755 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:44.310848 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:44.311663 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:44.311669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:44.311673 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:44.311690 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:44.311737 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:44.311757 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:44.311889 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:44.311928 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 09:19:44.311979 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:19:44.313956 20020 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3829c88a9e4e93b35b87b8fa28830d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } }
I20260811 09:19:44.314286 20020 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:44.314392 20020 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3829c88a9e4e93b35b87b8fa28830d, State: Initialized, Role: FOLLOWER
I20260811 09:19:44.314937 20020 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3829c88a9e4e93b35b87b8fa28830d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } }
I20260811 09:19:44.315157 20020 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:44.315284 20020 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:44.315439 20020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:44.317797 20020 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3829c88a9e4e93b35b87b8fa28830d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } }
I20260811 09:19:44.318264 20020 leader_election.cc:302] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec3829c88a9e4e93b35b87b8fa28830d; no voters:
I20260811 09:19:44.318790 20020 leader_election.cc:288] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:44.318910 20036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:44.319516 20036 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [term 1 LEADER]: Becoming Leader. State: Replica: ec3829c88a9e4e93b35b87b8fa28830d, State: Running, Role: LEADER
I20260811 09:19:44.320084 20036 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3829c88a9e4e93b35b87b8fa28830d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } }
I20260811 09:19:44.320684 20020 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:44.323052 20039 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec3829c88a9e4e93b35b87b8fa28830d. Latest consensus state: current_term: 1 leader_uuid: "ec3829c88a9e4e93b35b87b8fa28830d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3829c88a9e4e93b35b87b8fa28830d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } } }
I20260811 09:19:44.323796 20039 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:44.323411 20037 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec3829c88a9e4e93b35b87b8fa28830d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3829c88a9e4e93b35b87b8fa28830d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33637 } } }
I20260811 09:19:44.324393 20037 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:44.325433 20050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:44.329993 20050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:44.338773 20050 catalog_manager.cc:1347] Generated new cluster ID: 6e7f5f85ec6040b8b5fb54e01bdf6a1b
I20260811 09:19:44.338943 20050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:44.358590 20050 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:44.359078 20050 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:44.366070 20050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec3829c88a9e4e93b35b87b8fa28830d: Generated new TSK 0
I20260811 09:19:44.367022 20050 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:44.373872 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19903
W20260811 09:19:44.374882 20010 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52289
W20260811 09:19:44.375481 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 19903: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:19:44.375607 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 19903
2026-08-11T09:19:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2126 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:44 dist-test-slave-ncp9 krb5kdc[20074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:44 dist-test-slave-ncp9 krb5kdc[20074](info): set up 2 sockets
Aug 11 09:19:44 dist-test-slave-ncp9 krb5kdc[20074](info): commencing operation
krb5kdc: starting...
W20260811 09:19:45.627285 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:45 dist-test-slave-ncp9 krb5kdc[20074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:45.946552 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:46.284832 20093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:46.285135 20093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:46.285235 20093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:46.292356 20093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:46.292474 20093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:46.292529 20093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:46.292588 20093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:46.292631 20093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:46.302392 20093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:46.303611 20093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:46.304859 20093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:46.316313 20100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:46.316345 20109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:46.316401 20101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:46.317438 20093 server_base.cc:1034] running on GCE node
I20260811 09:19:46.318171 20093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:46.319551 20093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:46.320762 20093 hybrid_clock.cc:648] HybridClock initialized: now 1786439986320712 us; error 50 us; skew 500 ppm
I20260811 09:19:46.321131 20093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:46 dist-test-slave-ncp9 krb5kdc[20074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439986, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:46.324115 20093 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:46.325613 20093 webserver.cc:460] Webserver started at http://127.0.0.1:33647/ using document root <none> and password file <none>
I20260811 09:19:46.326243 20093 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:46.326364 20093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:46.326665 20093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:46.329646 20093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c"
format_stamp: "Formatted at 2026-08-11 09:19:46 on dist-test-slave-ncp9"
server_key: "cacccdc1e57d794b6e64e742d05991a9"
server_key_iv: "7759d849555f3454a00d8820932c60a4"
server_key_version: "encryptionkey@0"
I20260811 09:19:46.330394 20093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c"
format_stamp: "Formatted at 2026-08-11 09:19:46 on dist-test-slave-ncp9"
server_key: "cacccdc1e57d794b6e64e742d05991a9"
server_key_iv: "7759d849555f3454a00d8820932c60a4"
server_key_version: "encryptionkey@0"
I20260811 09:19:46.335438 20093 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:19:46.338819 20124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:46.340152 20093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:19:46.340320 20093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c"
format_stamp: "Formatted at 2026-08-11 09:19:46 on dist-test-slave-ncp9"
server_key: "cacccdc1e57d794b6e64e742d05991a9"
server_key_iv: "7759d849555f3454a00d8820932c60a4"
server_key_version: "encryptionkey@0"
I20260811 09:19:46.340513 20093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:46.365957 20093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:46.369943 20093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:46.370220 20093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:46.387979 20093 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35201
I20260811 09:19:46.388007 20218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35201 every 8 connection(s)
I20260811 09:19:46.389407 20093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:46.393071 20220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:46.395669 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 20093
I20260811 09:19:46.395846 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:46.396284 17473 external_mini_cluster.cc:1428] Setting key e0e6e7ebcf575361444ecd68fa73bb83
I20260811 09:19:46.404680 20220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c: Bootstrap starting.
I20260811 09:19:46.408383 20220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:46.409714 20220 log.cc:824] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:19:46 dist-test-slave-ncp9 krb5kdc[20074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:19:46.412880 20220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c: No bootstrap required, opened a new log
I20260811 09:19:46.421239 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:46.398238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:38983 (local address 127.0.0.1:35201)
0811 09:19:46.398998 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:46.399009 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:46.399050 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:19:46.400695 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:46.400718 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:46.401168 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:46.401492 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:46.403261 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:46.404563 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:46.406323 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:46.407030 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:46.407173 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:46.412234 (+ 5061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:46.412247 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:46.412267 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:46.412310 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:46.416244 (+ 3934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:46.417203 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:46.417215 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:46.417224 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:46.417443 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:46.417923 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:46.417928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:46.417935 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:46.418669 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:46.418997 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:46.419444 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 09:19:46.419793 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":331,"threads_started":1}
I20260811 09:19:46.423261 20220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35201 } }
I20260811 09:19:46.423727 20220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:46.423830 20220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e17fd7f5c8d4dd98c7abf75c2f45c3c, State: Initialized, Role: FOLLOWER
I20260811 09:19:46.424477 20220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35201 } }
I20260811 09:19:46.424742 20220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:46.424880 20220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:46.425081 20220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:46.425824 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:46.428282 20220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35201 } }
I20260811 09:19:46.428855 20220 leader_election.cc:302] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e17fd7f5c8d4dd98c7abf75c2f45c3c; no voters:
I20260811 09:19:46.429579 20220 leader_election.cc:288] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:46.429723 20230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:46.430472 20230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [term 1 LEADER]: Becoming Leader. State: Replica: 8e17fd7f5c8d4dd98c7abf75c2f45c3c, State: Running, Role: LEADER
I20260811 09:19:46.431181 20230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35201 } }
I20260811 09:19:46.435088 20230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e17fd7f5c8d4dd98c7abf75c2f45c3c. Latest consensus state: current_term: 1 leader_uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35201 } } }
I20260811 09:19:46.436151 20230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:46.438061 20238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e17fd7f5c8d4dd98c7abf75c2f45c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35201 } } }
I20260811 09:19:46.438494 20238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:46.440011 20220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:46.440249 20243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:46.449731 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:46.431322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:53893 (local address 127.0.0.1:35201)
0811 09:19:46.431500 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:46.431506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:46.432100 (+ 594us) server_negotiation.cc:407] Connection header received
0811 09:19:46.432378 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:46.432384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:46.432486 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:46.432605 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:46.434538 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:46.435813 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:46.437843 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:46.438519 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:46.438660 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:46.442046 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:46.442053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:46.442058 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:46.442099 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:46.446292 (+ 4193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:46.447369 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:46.447375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:46.447379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:46.447522 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:46.448042 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:46.448047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:46.448050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:46.448452 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:46.448740 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:46.449194 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 09:19:46.449378 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:19:46.455361 20243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:46.465313 20243 catalog_manager.cc:1347] Generated new cluster ID: 95bc0dbd77b14b388419a1f6e0d3423e
I20260811 09:19:46.465430 20243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:46.478319 20243 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:46.479624 20243 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:46.489670 20243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e17fd7f5c8d4dd98c7abf75c2f45c3c: Generated new TSK 0
I20260811 09:19:46.490754 20243 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:46.499719 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20093
I20260811 09:19:46.500981 20213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:46.590842 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 20093
W20260811 09:19:46.614291 20072 connection.cc:535] client connection to 127.0.0.1:35201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35201): Connection reset by peer (error 104)
2026-08-11T09:19:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2212 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:46 dist-test-slave-ncp9 krb5kdc[20273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:46 dist-test-slave-ncp9 krb5kdc[20273](info): set up 2 sockets
Aug 11 09:19:46 dist-test-slave-ncp9 krb5kdc[20273](info): commencing operation
krb5kdc: starting...
W20260811 09:19:47.855517 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.004s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:48 dist-test-slave-ncp9 krb5kdc[20273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:48.186096 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:48.541016 20293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:48.541317 20293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:48.541456 20293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:48.548606 20293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:48.548727 20293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:48.548811 20293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:48.548916 20293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:48.548976 20293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:48.558686 20293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:48.560173 20293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:48.561560 20293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:48.573215 20302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:48.573251 20307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:48.573319 20304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:48.574175 20293 server_base.cc:1034] running on GCE node
I20260811 09:19:48.574944 20293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:48.576314 20293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:48.577510 20293 hybrid_clock.cc:648] HybridClock initialized: now 1786439988577451 us; error 52 us; skew 500 ppm
I20260811 09:19:48.577853 20293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:48 dist-test-slave-ncp9 krb5kdc[20273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439988, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:48.580873 20293 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:48.582381 20293 webserver.cc:460] Webserver started at http://127.0.0.1:40139/ using document root <none> and password file <none>
I20260811 09:19:48.583045 20293 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:48.583165 20293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:48.583447 20293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:48.586460 20293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "d7e3fd72487f4959b1d8956a33d2cd3b"
format_stamp: "Formatted at 2026-08-11 09:19:48 on dist-test-slave-ncp9"
server_key: "06fd21cbf49fcba591462a78107f24d8"
server_key_iv: "b46a2247d86954a09de806a85875560f"
server_key_version: "encryptionkey@0"
I20260811 09:19:48.587230 20293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "d7e3fd72487f4959b1d8956a33d2cd3b"
format_stamp: "Formatted at 2026-08-11 09:19:48 on dist-test-slave-ncp9"
server_key: "06fd21cbf49fcba591462a78107f24d8"
server_key_iv: "b46a2247d86954a09de806a85875560f"
server_key_version: "encryptionkey@0"
I20260811 09:19:48.592178 20293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 09:19:48.595485 20320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:48.596736 20293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:19:48.596916 20293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "d7e3fd72487f4959b1d8956a33d2cd3b"
format_stamp: "Formatted at 2026-08-11 09:19:48 on dist-test-slave-ncp9"
server_key: "06fd21cbf49fcba591462a78107f24d8"
server_key_iv: "b46a2247d86954a09de806a85875560f"
server_key_version: "encryptionkey@0"
I20260811 09:19:48.597105 20293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:48.629021 20293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:48.633383 20293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:48.633667 20293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:48.651218 20293 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43975
I20260811 09:19:48.651262 20406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43975 every 8 connection(s)
I20260811 09:19:48.652642 20293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:48.656368 20408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:48.659117 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 20293
I20260811 09:19:48.659307 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:48.659865 17473 external_mini_cluster.cc:1428] Setting key 2cd70be1deb5e18fbb6c00523a550ef2
I20260811 09:19:48.668371 20408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b: Bootstrap starting.
I20260811 09:19:48.671854 20408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:48.673066 20408 log.cc:824] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:48.673707 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:48.661770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:44079 (local address 127.0.0.1:43975)
0811 09:19:48.662545 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:48.662556 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:48.662604 (+ 48us) server_negotiation.cc:407] Connection header received
0811 09:19:48.664248 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:48.664271 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:48.664662 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:48.665045 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:48.665641 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:48.666958 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:48.668812 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:48.669625 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:48.669785 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:48.671132 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:48.671140 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:48.671154 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:48.671715 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:48.671799 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:48.671828 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:48.671991 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:48.672294 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 09:19:48.672517 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":356,"threads_started":1}
I20260811 09:19:48.676195 20408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b: No bootstrap required, opened a new log
I20260811 09:19:48.677742 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:48.683794 20408 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e3fd72487f4959b1d8956a33d2cd3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } }
I20260811 09:19:48.684319 20408 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:48.684437 20408 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e3fd72487f4959b1d8956a33d2cd3b, State: Initialized, Role: FOLLOWER
I20260811 09:19:48.685106 20408 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e3fd72487f4959b1d8956a33d2cd3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } }
I20260811 09:19:48.685353 20408 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:48.685487 20408 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:48.685642 20408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:48.688257 20408 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e3fd72487f4959b1d8956a33d2cd3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } }
I20260811 09:19:48.688755 20408 leader_election.cc:302] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7e3fd72487f4959b1d8956a33d2cd3b; no voters:
I20260811 09:19:48.689513 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:48.682301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:42997 (local address 127.0.0.1:43975)
0811 09:19:48.682479 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:48.682485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:48.682901 (+ 416us) server_negotiation.cc:407] Connection header received
0811 09:19:48.683061 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:48.683067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:48.683144 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:48.683245 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:48.683864 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:48.684998 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:48.686560 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:48.687282 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:48.687412 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:48.688597 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:48.688605 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:48.688608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:48.688629 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:48.688693 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:48.688712 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:48.688831 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:48.689095 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 09:19:48.689153 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:19:48.689484 20408 leader_election.cc:288] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:48.689633 20424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:48.691099 20424 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [term 1 LEADER]: Becoming Leader. State: Replica: d7e3fd72487f4959b1d8956a33d2cd3b, State: Running, Role: LEADER
I20260811 09:19:48.691668 20424 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e3fd72487f4959b1d8956a33d2cd3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } }
I20260811 09:19:48.691953 20408 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:48.694471 20426 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7e3fd72487f4959b1d8956a33d2cd3b. Latest consensus state: current_term: 1 leader_uuid: "d7e3fd72487f4959b1d8956a33d2cd3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e3fd72487f4959b1d8956a33d2cd3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } } }
I20260811 09:19:48.694882 20426 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:48.695451 20425 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7e3fd72487f4959b1d8956a33d2cd3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e3fd72487f4959b1d8956a33d2cd3b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } } }
I20260811 09:19:48.695832 20425 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:48.700094 20432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:48.704779 20432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:48.713266 20432 catalog_manager.cc:1347] Generated new cluster ID: e49417632d574b36b43e1efb80322ed4
I20260811 09:19:48.713383 20432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:48.725016 20432 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:48.726294 20432 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:48.732776 20432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7e3fd72487f4959b1d8956a33d2cd3b: Generated new TSK 0
I20260811 09:19:48.733835 20432 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:48.746973 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20293
W20260811 09:19:48.748239 20402 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.213:44079
W20260811 09:19:48.749202 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20293: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:19:48.749361 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 20293
W20260811 09:19:48.774744 20268 connection.cc:535] client connection to 127.0.0.1:43975 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43975): Connection reset by peer (error 104)
2026-08-11T09:19:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2160 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:48 dist-test-slave-ncp9 krb5kdc[20459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:48 dist-test-slave-ncp9 krb5kdc[20459](info): set up 2 sockets
Aug 11 09:19:48 dist-test-slave-ncp9 krb5kdc[20459](info): commencing operation
krb5kdc: starting...
W20260811 09:19:50.006055 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:50 dist-test-slave-ncp9 krb5kdc[20459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:50.348245 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:50.687098 20483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:50.687397 20483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:50.687532 20483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:50.694664 20483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:50.694801 20483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:50.694887 20483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:50.694980 20483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:50.695046 20483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:50.705040 20483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:50.706403 20483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:50.707731 20483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:50.719297 20494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:50.719228 20499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:50.719347 20495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:50.720353 20483 server_base.cc:1034] running on GCE node
I20260811 09:19:50.721261 20483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:50.722733 20483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:50.723980 20483 hybrid_clock.cc:648] HybridClock initialized: now 1786439990723930 us; error 34 us; skew 500 ppm
I20260811 09:19:50.724352 20483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:50 dist-test-slave-ncp9 krb5kdc[20459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439990, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:50.727491 20483 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:50.729061 20483 webserver.cc:460] Webserver started at http://127.0.0.1:33161/ using document root <none> and password file <none>
I20260811 09:19:50.729748 20483 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:50.729866 20483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:50.730162 20483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:50.733173 20483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "9bbdc47d4c104d839fa875295e3e7ad0"
format_stamp: "Formatted at 2026-08-11 09:19:50 on dist-test-slave-ncp9"
server_key: "89141a5f50f17d81fb128ce24546d783"
server_key_iv: "f51df66a728018ca94a3a26b1f91e7c9"
server_key_version: "encryptionkey@0"
I20260811 09:19:50.733933 20483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "9bbdc47d4c104d839fa875295e3e7ad0"
format_stamp: "Formatted at 2026-08-11 09:19:50 on dist-test-slave-ncp9"
server_key: "89141a5f50f17d81fb128ce24546d783"
server_key_iv: "f51df66a728018ca94a3a26b1f91e7c9"
server_key_version: "encryptionkey@0"
I20260811 09:19:50.738848 20483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:19:50.742234 20513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:50.743508 20483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:19:50.743670 20483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "9bbdc47d4c104d839fa875295e3e7ad0"
format_stamp: "Formatted at 2026-08-11 09:19:50 on dist-test-slave-ncp9"
server_key: "89141a5f50f17d81fb128ce24546d783"
server_key_iv: "f51df66a728018ca94a3a26b1f91e7c9"
server_key_version: "encryptionkey@0"
I20260811 09:19:50.743841 20483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:50.749586 20483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:50.753309 20483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:50.753588 20483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:50.770951 20483 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45329
I20260811 09:19:50.770984 20608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45329 every 8 connection(s)
I20260811 09:19:50.772275 20483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:50.775892 20613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:50.777877 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 20483
I20260811 09:19:50.778065 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:50.778594 17473 external_mini_cluster.cc:1428] Setting key a33e30757adb57abd138a6c86f6cfda9
I20260811 09:19:50.787842 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:50.780541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:39863 (local address 127.0.0.1:45329)
0811 09:19:50.781459 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:50.781476 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:50.781790 (+ 314us) server_negotiation.cc:407] Connection header received
0811 09:19:50.784073 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:50.784099 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:50.784506 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:50.784984 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:50.785945 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:50.785954 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:50.785971 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:50.785988 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:50.786057 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:50.786095 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:50.786327 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:50.786461 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 09:19:50.786668 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":400,"threads_started":1}
I20260811 09:19:50.789072 20613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0: Bootstrap starting.
I20260811 09:19:50.791352 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:50.792306 20613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:50.793545 20613 log.cc:824] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:50.796713 20613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0: No bootstrap required, opened a new log
I20260811 09:19:50.798971 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:50.796511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:39421 (local address 127.0.0.1:45329)
0811 09:19:50.796687 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:50.796692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:50.797105 (+ 413us) server_negotiation.cc:407] Connection header received
0811 09:19:50.797285 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:50.797289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:50.797336 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:50.797416 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:50.798340 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:50.798344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:50.798351 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:50.798366 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:50.798410 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:50.798425 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:50.798524 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:50.798650 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 09:19:50.798711 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:19:50.804534 20613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbdc47d4c104d839fa875295e3e7ad0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } }
I20260811 09:19:50.804924 20613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:50.805032 20613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bbdc47d4c104d839fa875295e3e7ad0, State: Initialized, Role: FOLLOWER
I20260811 09:19:50.805581 20613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbdc47d4c104d839fa875295e3e7ad0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } }
I20260811 09:19:50.805795 20613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:50.805925 20613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:50.806078 20613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:50.808408 20613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbdc47d4c104d839fa875295e3e7ad0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } }
I20260811 09:19:50.808883 20613 leader_election.cc:302] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bbdc47d4c104d839fa875295e3e7ad0; no voters:
I20260811 09:19:50.809475 20613 leader_election.cc:288] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:50.809602 20627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:50.810266 20627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [term 1 LEADER]: Becoming Leader. State: Replica: 9bbdc47d4c104d839fa875295e3e7ad0, State: Running, Role: LEADER
I20260811 09:19:50.810791 20627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbdc47d4c104d839fa875295e3e7ad0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } }
I20260811 09:19:50.811394 20613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:50.813603 20628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bbdc47d4c104d839fa875295e3e7ad0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbdc47d4c104d839fa875295e3e7ad0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } } }
I20260811 09:19:50.813994 20628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:50.814922 20631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bbdc47d4c104d839fa875295e3e7ad0. Latest consensus state: current_term: 1 leader_uuid: "9bbdc47d4c104d839fa875295e3e7ad0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bbdc47d4c104d839fa875295e3e7ad0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45329 } } }
I20260811 09:19:50.815290 20631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:50.822083 20638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:50.826979 20638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:50.835764 20638 catalog_manager.cc:1347] Generated new cluster ID: 61e35914f64e454f8bedca9f6e9fcec8
I20260811 09:19:50.835865 20638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:50.860713 20638 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:50.861179 20638 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:50.873843 20638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bbdc47d4c104d839fa875295e3e7ad0: Generated new TSK 0
I20260811 09:19:50.874670 20638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:50.888221 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20483
W20260811 09:19:50.889376 20599 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.213:39863
W20260811 09:19:50.890070 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20483: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:19:50.890223 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 20483
2026-08-11T09:19:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2139 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:51 dist-test-slave-ncp9 krb5kdc[20660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:51 dist-test-slave-ncp9 krb5kdc[20660](info): set up 2 sockets
Aug 11 09:19:51 dist-test-slave-ncp9 krb5kdc[20660](info): commencing operation
krb5kdc: starting...
W20260811 09:19:52.166757 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.001s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:52 dist-test-slave-ncp9 krb5kdc[20660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:52.504478 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:52.848086 20684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:52.848361 20684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:52.848501 20684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:52.855554 20684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:52.855661 20684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:52.855736 20684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:52.855814 20684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:52.855876 20684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:52.865774 20684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37039
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:52.867097 20684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:52.868422 20684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:52.879825 20699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:52.879896 20692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:52.879902 20695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:52.881178 20684 server_base.cc:1034] running on GCE node
I20260811 09:19:52.881942 20684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:52.883278 20684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:52.884488 20684 hybrid_clock.cc:648] HybridClock initialized: now 1786439992884431 us; error 42 us; skew 500 ppm
I20260811 09:19:52.884841 20684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:52 dist-test-slave-ncp9 krb5kdc[20660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439992, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:52.887840 20684 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:52.889345 20684 webserver.cc:460] Webserver started at http://127.0.0.1:35365/ using document root <none> and password file <none>
I20260811 09:19:52.889976 20684 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:52.890107 20684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:52.890398 20684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:52.893383 20684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "3b67e7be3da04d9abec0eebc0a276932"
format_stamp: "Formatted at 2026-08-11 09:19:52 on dist-test-slave-ncp9"
server_key: "2a5c9e6a0d3c05322651c214a1d6ce05"
server_key_iv: "09f39b5823e0b709f81144a82eb60e9d"
server_key_version: "encryptionkey@0"
I20260811 09:19:52.894153 20684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "3b67e7be3da04d9abec0eebc0a276932"
format_stamp: "Formatted at 2026-08-11 09:19:52 on dist-test-slave-ncp9"
server_key: "2a5c9e6a0d3c05322651c214a1d6ce05"
server_key_iv: "09f39b5823e0b709f81144a82eb60e9d"
server_key_version: "encryptionkey@0"
I20260811 09:19:52.899190 20684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:19:52.902606 20716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:52.903831 20684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:19:52.904006 20684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "3b67e7be3da04d9abec0eebc0a276932"
format_stamp: "Formatted at 2026-08-11 09:19:52 on dist-test-slave-ncp9"
server_key: "2a5c9e6a0d3c05322651c214a1d6ce05"
server_key_iv: "09f39b5823e0b709f81144a82eb60e9d"
server_key_version: "encryptionkey@0"
I20260811 09:19:52.904181 20684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:52.921672 20684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:52.925557 20684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:52.925824 20684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:52.943356 20684 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37039
I20260811 09:19:52.943374 20794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37039 every 8 connection(s)
I20260811 09:19:52.944730 20684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:52.946085 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 20684
I20260811 09:19:52.946290 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:52.946839 17473 external_mini_cluster.cc:1428] Setting key 0076b44027162f180c7be83e8bfce42f
I20260811 09:19:52.949455 20795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:19:52 dist-test-slave-ncp9 krb5kdc[20660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:19:52.962360 20795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932: Bootstrap starting.
I20260811 09:19:52.966430 20795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:52.967674 20795 log.cc:824] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:52.970628 20795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932: No bootstrap required, opened a new log
I20260811 09:19:52.971457 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:52.948641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:34735 (local address 127.0.0.1:37039)
0811 09:19:52.949414 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:52.949429 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:52.949485 (+ 56us) server_negotiation.cc:407] Connection header received
0811 09:19:52.951746 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:52.951772 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:52.952266 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:52.952706 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:52.954218 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:52.956219 (+ 2001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:52.957567 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:52.958590 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:52.958775 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:52.963182 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:52.963193 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:52.963211 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:52.963261 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:52.967057 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:52.967935 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:52.967942 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:52.967948 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:52.968146 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:52.968629 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:52.968633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:52.968635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:52.969239 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:52.969540 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:52.970085 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 09:19:52.970387 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":336,"threads_started":1}
I20260811 09:19:52.975618 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:52.979115 20795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b67e7be3da04d9abec0eebc0a276932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } }
I20260811 09:19:52.979472 20795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:52.979561 20795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b67e7be3da04d9abec0eebc0a276932, State: Initialized, Role: FOLLOWER
I20260811 09:19:52.980160 20795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b67e7be3da04d9abec0eebc0a276932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } }
I20260811 09:19:52.980407 20795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:52.980540 20795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:52.980731 20795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:52.982961 20795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b67e7be3da04d9abec0eebc0a276932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } }
I20260811 09:19:52.983515 20795 leader_election.cc:302] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b67e7be3da04d9abec0eebc0a276932; no voters:
I20260811 09:19:52.984129 20795 leader_election.cc:288] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:52.984241 20816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:52.986450 20795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:52.989267 20816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [term 1 LEADER]: Becoming Leader. State: Replica: 3b67e7be3da04d9abec0eebc0a276932, State: Running, Role: LEADER
I20260811 09:19:52.990026 20816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b67e7be3da04d9abec0eebc0a276932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } }
I20260811 09:19:52.993093 20817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b67e7be3da04d9abec0eebc0a276932" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b67e7be3da04d9abec0eebc0a276932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } } }
I20260811 09:19:52.993517 20817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:52.994005 20822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b67e7be3da04d9abec0eebc0a276932. Latest consensus state: current_term: 1 leader_uuid: "3b67e7be3da04d9abec0eebc0a276932" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b67e7be3da04d9abec0eebc0a276932" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } } }
I20260811 09:19:52.994397 20822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:52.995114 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:52.979966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:50363 (local address 127.0.0.1:37039)
0811 09:19:52.980167 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:52.980173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:52.980655 (+ 482us) server_negotiation.cc:407] Connection header received
0811 09:19:52.980832 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:52.980838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:52.980944 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:52.981053 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:52.982521 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:52.983339 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:52.984774 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:52.985368 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:52.985440 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:52.988046 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:52.988052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:52.988056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:52.988087 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:52.991254 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:52.992153 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:52.992159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:52.992163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:52.992423 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:52.993107 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:52.993112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:52.993116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:52.993535 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:52.993831 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:52.994560 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 09:19:52.994744 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:19:53.002166 20836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:53.007807 20836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:53.016800 20836 catalog_manager.cc:1347] Generated new cluster ID: 2f22cd64da7b4527a5a6773693c206be
I20260811 09:19:53.016986 20836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:53.035359 20836 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:53.037294 20836 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:53.048417 20836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b67e7be3da04d9abec0eebc0a276932: Generated new TSK 0
I20260811 09:19:53.049619 20836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:53.063167 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20684
I20260811 09:19:53.064332 20788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:53.153738 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 20684
W20260811 09:19:53.178763 20658 connection.cc:535] client connection to 127.0.0.1:37039 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37039): Connection reset by peer (error 104)
2026-08-11T09:19:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2264 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:53 dist-test-slave-ncp9 krb5kdc[20865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:53 dist-test-slave-ncp9 krb5kdc[20865](info): set up 2 sockets
Aug 11 09:19:53 dist-test-slave-ncp9 krb5kdc[20865](info): commencing operation
krb5kdc: starting...
W20260811 09:19:54.409353 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.003s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:54 dist-test-slave-ncp9 krb5kdc[20865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:54.724498 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45623
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:55.060716 20886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:55.061022 20886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:55.061136 20886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:55.068337 20886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:55.068447 20886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:55.068500 20886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:55.068559 20886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:55.068603 20886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:55.078339 20886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45623
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:55.079608 20886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:55.080862 20886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:55.092227 20898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:55.092295 20893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:55.092268 20891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:55.093397 20886 server_base.cc:1034] running on GCE node
I20260811 09:19:55.094117 20886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:55.095420 20886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:55.096626 20886 hybrid_clock.cc:648] HybridClock initialized: now 1786439995096585 us; error 25 us; skew 500 ppm
I20260811 09:19:55.097003 20886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:55 dist-test-slave-ncp9 krb5kdc[20865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:55.099960 20886 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:55.101382 20886 webserver.cc:460] Webserver started at http://127.0.0.1:40297/ using document root <none> and password file <none>
I20260811 09:19:55.102013 20886 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:55.102114 20886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:55.102392 20886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:55.105353 20886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "9294b988510c483bbad75e67d0bc41a6"
format_stamp: "Formatted at 2026-08-11 09:19:55 on dist-test-slave-ncp9"
server_key: "0a5dffe2780833b70219e4f221d67065"
server_key_iv: "4167f0d4234c1b31746645d52e112a76"
server_key_version: "encryptionkey@0"
I20260811 09:19:55.106061 20886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "9294b988510c483bbad75e67d0bc41a6"
format_stamp: "Formatted at 2026-08-11 09:19:55 on dist-test-slave-ncp9"
server_key: "0a5dffe2780833b70219e4f221d67065"
server_key_iv: "4167f0d4234c1b31746645d52e112a76"
server_key_version: "encryptionkey@0"
I20260811 09:19:55.111093 20886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:19:55.114429 20912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:55.115692 20886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:19:55.115844 20886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "9294b988510c483bbad75e67d0bc41a6"
format_stamp: "Formatted at 2026-08-11 09:19:55 on dist-test-slave-ncp9"
server_key: "0a5dffe2780833b70219e4f221d67065"
server_key_iv: "4167f0d4234c1b31746645d52e112a76"
server_key_version: "encryptionkey@0"
I20260811 09:19:55.116021 20886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:55.128088 20886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:55.131785 20886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:55.132027 20886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:55.148823 20886 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45623
I20260811 09:19:55.148845 20996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45623 every 8 connection(s)
I20260811 09:19:55.150206 20886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:55.153874 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 20886
I20260811 09:19:55.154096 20998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:55.154112 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:55.154681 17473 external_mini_cluster.cc:1428] Setting key 2077d5c85222199d2833ced80bfc5a4f
I20260811 09:19:55.168874 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:55.156398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:42259 (local address 127.0.0.1:45623)
0811 09:19:55.157218 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:55.157233 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:55.157285 (+ 52us) server_negotiation.cc:407] Connection header received
0811 09:19:55.159066 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:55.159094 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:55.159594 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:55.160026 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:55.160544 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:55.162559 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:55.163850 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:55.164878 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:55.165096 (+ 218us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:55.166199 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:55.166214 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:55.166232 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:55.166943 (+ 711us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:55.167046 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:55.167077 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:55.167222 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:55.167471 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 09:19:55.167630 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":297,"threads_started":1}
I20260811 09:19:55.169103 20998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6: Bootstrap starting.
I20260811 09:19:55.172664 20998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:55.173031 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:55.173760 20998 log.cc:824] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:55.176661 20998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6: No bootstrap required, opened a new log
I20260811 09:19:55.183681 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:55.177573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:39059 (local address 127.0.0.1:45623)
0811 09:19:55.177738 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:55.177744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:55.178094 (+ 350us) server_negotiation.cc:407] Connection header received
0811 09:19:55.178263 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:55.178266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:55.178333 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:55.178422 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:55.178957 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:55.179768 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:55.181076 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:55.181560 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:55.181685 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:55.182942 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:55.182950 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:55.182954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:55.182972 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:55.183013 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:55.183028 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:55.183149 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:55.183348 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 09:19:55.183403 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:19:55.184983 20998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9294b988510c483bbad75e67d0bc41a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45623 } }
I20260811 09:19:55.185380 20998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:55.185487 20998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9294b988510c483bbad75e67d0bc41a6, State: Initialized, Role: FOLLOWER
I20260811 09:19:55.186056 20998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9294b988510c483bbad75e67d0bc41a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45623 } }
I20260811 09:19:55.186280 20998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:55.186414 20998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:55.186578 20998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:55.188710 20998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9294b988510c483bbad75e67d0bc41a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45623 } }
I20260811 09:19:55.189225 20998 leader_election.cc:302] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9294b988510c483bbad75e67d0bc41a6; no voters:
I20260811 09:19:55.189790 20998 leader_election.cc:288] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:55.189919 21013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:55.191004 21013 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [term 1 LEADER]: Becoming Leader. State: Replica: 9294b988510c483bbad75e67d0bc41a6, State: Running, Role: LEADER
I20260811 09:19:55.191618 21013 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9294b988510c483bbad75e67d0bc41a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45623 } }
I20260811 09:19:55.191977 20998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:55.194590 21018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9294b988510c483bbad75e67d0bc41a6. Latest consensus state: current_term: 1 leader_uuid: "9294b988510c483bbad75e67d0bc41a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9294b988510c483bbad75e67d0bc41a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45623 } } }
I20260811 09:19:55.195031 21018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:55.194573 21016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9294b988510c483bbad75e67d0bc41a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9294b988510c483bbad75e67d0bc41a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45623 } } }
I20260811 09:19:55.195281 21016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:55.196498 21027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:55.201144 21027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:55.209348 21027 catalog_manager.cc:1347] Generated new cluster ID: 62f5e361643c4890b9e9c0604c3ac092
I20260811 09:19:55.209450 21027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:55.222908 21027 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:55.224676 21027 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:55.236361 21027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9294b988510c483bbad75e67d0bc41a6: Generated new TSK 0
I20260811 09:19:55.237424 21027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:55.245725 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20886
W20260811 09:19:55.246877 20990 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.213:42259
W20260811 09:19:55.247620 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 20886: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:19:55.247781 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 20886
W20260811 09:19:55.272200 20863 connection.cc:535] client connection to 127.0.0.1:45623 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45623): Connection reset by peer (error 104)
2026-08-11T09:19:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2094 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:55 dist-test-slave-ncp9 krb5kdc[21048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:55 dist-test-slave-ncp9 krb5kdc[21048](info): set up 2 sockets
Aug 11 09:19:55 dist-test-slave-ncp9 krb5kdc[21048](info): commencing operation
krb5kdc: starting...
W20260811 09:19:56.500205 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.003s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:56 dist-test-slave-ncp9 krb5kdc[21048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:19:56.822469 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34537
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:57.158197 21078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:57.158470 21078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:57.158612 21078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:57.165693 21078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:57.165795 21078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:57.165874 21078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:57.165963 21078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:57.166023 21078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:57.175608 21078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34537
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:57.176867 21078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:57.178165 21078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:57.189349 21084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:57.189309 21086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:57.189539 21089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:57.190337 21078 server_base.cc:1034] running on GCE node
I20260811 09:19:57.191076 21078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:57.192405 21078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:57.193610 21078 hybrid_clock.cc:648] HybridClock initialized: now 1786439997193562 us; error 34 us; skew 500 ppm
I20260811 09:19:57.193948 21078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:57 dist-test-slave-ncp9 krb5kdc[21048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439997, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:57.196933 21078 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:19:57.198351 21078 webserver.cc:460] Webserver started at http://127.0.0.1:43577/ using document root <none> and password file <none>
I20260811 09:19:57.199031 21078 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:57.199158 21078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:57.199455 21078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:57.202512 21078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "61d78baf77ef4d34af95abe534e366d4"
format_stamp: "Formatted at 2026-08-11 09:19:57 on dist-test-slave-ncp9"
server_key: "0751fc87ef62266338de1cae21c5f3aa"
server_key_iv: "c69777432fe7bf42c0ac35de1f6f5717"
server_key_version: "encryptionkey@0"
I20260811 09:19:57.203270 21078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "61d78baf77ef4d34af95abe534e366d4"
format_stamp: "Formatted at 2026-08-11 09:19:57 on dist-test-slave-ncp9"
server_key: "0751fc87ef62266338de1cae21c5f3aa"
server_key_iv: "c69777432fe7bf42c0ac35de1f6f5717"
server_key_version: "encryptionkey@0"
I20260811 09:19:57.208316 21078 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:19:57.211642 21102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:57.212977 21078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:19:57.213133 21078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "61d78baf77ef4d34af95abe534e366d4"
format_stamp: "Formatted at 2026-08-11 09:19:57 on dist-test-slave-ncp9"
server_key: "0751fc87ef62266338de1cae21c5f3aa"
server_key_iv: "c69777432fe7bf42c0ac35de1f6f5717"
server_key_version: "encryptionkey@0"
I20260811 09:19:57.213315 21078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:57.219182 21078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:57.222905 21078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:57.223181 21078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:57.240353 21078 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34537
I20260811 09:19:57.240402 21185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34537 every 8 connection(s)
I20260811 09:19:57.241735 21078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:57.243209 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 21078
I20260811 09:19:57.243397 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:57.243970 17473 external_mini_cluster.cc:1428] Setting key 2d7bd6adc5480c4912f436840befd980
I20260811 09:19:57.246157 21187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:57.251410 21191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:57.245791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:53791 (local address 127.0.0.1:34537)
0811 09:19:57.246549 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:57.246560 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:57.246609 (+ 49us) server_negotiation.cc:407] Connection header received
0811 09:19:57.248202 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:57.248223 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:57.248548 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:57.248872 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:57.249919 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:57.249926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:57.249937 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:57.249951 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:57.250025 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:57.250044 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:57.250210 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:57.250306 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 09:19:57.250468 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":326,"threads_started":1}
I20260811 09:19:57.255090 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:57.257929 21187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4: Bootstrap starting.
I20260811 09:19:57.261317 21187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:57.261754 21191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:57.259353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:52141 (local address 127.0.0.1:34537)
0811 09:19:57.259507 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:57.259511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:57.259949 (+ 438us) server_negotiation.cc:407] Connection header received
0811 09:19:57.260109 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:57.260112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:57.260159 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:57.260235 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:57.261118 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:57.261125 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:57.261130 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:19:57.261148 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:57.261185 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:19:57.261200 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:57.261302 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:57.261416 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 09:19:57.261479 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:19:57.262513 21187 log.cc:824] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:57.265069 21187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4: No bootstrap required, opened a new log
I20260811 09:19:57.272444 21187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d78baf77ef4d34af95abe534e366d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34537 } }
I20260811 09:19:57.272814 21187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:57.272918 21187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d78baf77ef4d34af95abe534e366d4, State: Initialized, Role: FOLLOWER
I20260811 09:19:57.273494 21187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d78baf77ef4d34af95abe534e366d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34537 } }
I20260811 09:19:57.273721 21187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:57.273852 21187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:57.274004 21187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:57.276347 21187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d78baf77ef4d34af95abe534e366d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34537 } }
I20260811 09:19:57.276842 21187 leader_election.cc:302] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61d78baf77ef4d34af95abe534e366d4; no voters:
I20260811 09:19:57.277426 21187 leader_election.cc:288] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:57.277547 21201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:57.278151 21201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [term 1 LEADER]: Becoming Leader. State: Replica: 61d78baf77ef4d34af95abe534e366d4, State: Running, Role: LEADER
I20260811 09:19:57.278640 21201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d78baf77ef4d34af95abe534e366d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34537 } }
I20260811 09:19:57.279321 21187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:57.281562 21202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61d78baf77ef4d34af95abe534e366d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d78baf77ef4d34af95abe534e366d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34537 } } }
I20260811 09:19:57.281638 21203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61d78baf77ef4d34af95abe534e366d4. Latest consensus state: current_term: 1 leader_uuid: "61d78baf77ef4d34af95abe534e366d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d78baf77ef4d34af95abe534e366d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34537 } } }
I20260811 09:19:57.281958 21202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:57.281963 21203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:57.283622 21211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:57.288164 21211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:57.296115 21211 catalog_manager.cc:1347] Generated new cluster ID: 94ab72e5c5184e25b212692f45035b60
I20260811 09:19:57.296211 21211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:57.308936 21211 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:57.309340 21211 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:57.315846 21211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61d78baf77ef4d34af95abe534e366d4: Generated new TSK 0
I20260811 09:19:57.316738 21211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:57.328753 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21078
W20260811 09:19:57.329808 21177 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.213:53791
W20260811 09:19:57.330430 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21078: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:19:57.330564 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 21078
2026-08-11T09:19:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2080 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:57 dist-test-slave-ncp9 krb5kdc[21235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:57 dist-test-slave-ncp9 krb5kdc[21235](info): set up 2 sockets
Aug 11 09:19:57 dist-test-slave-ncp9 krb5kdc[21235](info): commencing operation
krb5kdc: starting...
W20260811 09:19:58.604812 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:19:58 dist-test-slave-ncp9 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:19:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:19:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:19:58.939661 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41483
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:19:59.279073 21255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:19:59.279351 21255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:19:59.279484 21255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:19:59.286616 21255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:19:59.286724 21255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:19:59.286804 21255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:19:59.286895 21255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:19:59.286952 21255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:19:59.296626 21255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41483
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:19:59.297912 21255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:19:59.299222 21255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:19:59.310739 21268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:59.310817 21272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:19:59.310791 21275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:19:59.311582 21255 server_base.cc:1034] running on GCE node
I20260811 09:19:59.312318 21255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:19:59.313771 21255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:19:59.314996 21255 hybrid_clock.cc:648] HybridClock initialized: now 1786439999314948 us; error 27 us; skew 500 ppm
I20260811 09:19:59.315346 21255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:19:59 dist-test-slave-ncp9 krb5kdc[21235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439999, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:19:59.323220 21255 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:19:59.324810 21255 webserver.cc:460] Webserver started at http://127.17.16.126:38177/ using document root <none> and password file <none>
I20260811 09:19:59.325508 21255 fs_manager.cc:359] Metadata directory not provided
I20260811 09:19:59.325645 21255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:19:59.325943 21255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:19:59.329049 21255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "94fb5ec76f084aa283b127a803b26f5d"
format_stamp: "Formatted at 2026-08-11 09:19:59 on dist-test-slave-ncp9"
server_key: "c67b84857aa9410855b089cde725ad88"
server_key_iv: "3b75885f1ec2661bc08574907924c1f5"
server_key_version: "encryptionkey@0"
I20260811 09:19:59.329821 21255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "94fb5ec76f084aa283b127a803b26f5d"
format_stamp: "Formatted at 2026-08-11 09:19:59 on dist-test-slave-ncp9"
server_key: "c67b84857aa9410855b089cde725ad88"
server_key_iv: "3b75885f1ec2661bc08574907924c1f5"
server_key_version: "encryptionkey@0"
I20260811 09:19:59.334935 21255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:19:59.338582 21286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:19:59.339963 21255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:19:59.340145 21255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "94fb5ec76f084aa283b127a803b26f5d"
format_stamp: "Formatted at 2026-08-11 09:19:59 on dist-test-slave-ncp9"
server_key: "c67b84857aa9410855b089cde725ad88"
server_key_iv: "3b75885f1ec2661bc08574907924c1f5"
server_key_version: "encryptionkey@0"
I20260811 09:19:59.340339 21255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:19:59.359278 21255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:19:59.363435 21255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:19:59.363718 21255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:19:59.381517 21255 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:41483
I20260811 09:19:59.381552 21377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:41483 every 8 connection(s)
I20260811 09:19:59.382908 21255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:19:59.386533 21378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:19:59.392796 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 21255
I20260811 09:19:59.393010 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:19:59.393579 17473 external_mini_cluster.cc:1428] Setting key ec51aeaf50836b227f9aa3e7cd0f87a2
I20260811 09:19:59.397316 21378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d: Bootstrap starting.
I20260811 09:19:59.400830 21378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d: Neither blocks nor log segments found. Creating new log.
I20260811 09:19:59.401895 21378 log.cc:824] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d: Log is configured to *not* fsync() on all Append() calls
I20260811 09:19:59.404826 21378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d: No bootstrap required, opened a new log
Aug 11 09:19:59 dist-test-slave-ncp9 krb5kdc[21235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:19:59.412789 21378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94fb5ec76f084aa283b127a803b26f5d" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41483 } }
I20260811 09:19:59.413195 21378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:19:59.413285 21378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94fb5ec76f084aa283b127a803b26f5d, State: Initialized, Role: FOLLOWER
I20260811 09:19:59.414004 21378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94fb5ec76f084aa283b127a803b26f5d" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41483 } }
I20260811 09:19:59.414312 21378 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:19:59.414475 21378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:19:59.414659 21378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:19:59.416787 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:59.395348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56271 (local address 127.17.16.126:41483)
0811 09:19:59.396156 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:59.396178 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:59.396224 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:19:59.397955 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:59.397979 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:59.398351 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:59.398665 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:59.400428 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:59.401807 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:59.403358 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:59.404006 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:59.404131 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:59.408828 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:59.408836 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:59.408851 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:59.408919 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:59.411985 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:59.412764 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:59.412770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:59.412781 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:59.412989 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:59.413471 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:59.413476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:59.413479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:59.414221 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:59.414551 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:59.415046 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 09:19:59.415395 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":347,"threads_started":1}
I20260811 09:19:59.417855 21378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94fb5ec76f084aa283b127a803b26f5d" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41483 } }
I20260811 09:19:59.418383 21378 leader_election.cc:302] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94fb5ec76f084aa283b127a803b26f5d; no voters:
I20260811 09:19:59.419016 21378 leader_election.cc:288] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:19:59.419122 21383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:19:59.420203 21383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [term 1 LEADER]: Becoming Leader. State: Replica: 94fb5ec76f084aa283b127a803b26f5d, State: Running, Role: LEADER
I20260811 09:19:59.420840 21383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94fb5ec76f084aa283b127a803b26f5d" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41483 } }
I20260811 09:19:59.421120 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:19:59.421162 21378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:19:59.428663 21387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94fb5ec76f084aa283b127a803b26f5d. Latest consensus state: current_term: 1 leader_uuid: "94fb5ec76f084aa283b127a803b26f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94fb5ec76f084aa283b127a803b26f5d" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41483 } } }
I20260811 09:19:59.429123 21387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:59.429708 21385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94fb5ec76f084aa283b127a803b26f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94fb5ec76f084aa283b127a803b26f5d" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41483 } } }
I20260811 09:19:59.431803 21385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d [sys.catalog]: This master's current role is: LEADER
I20260811 09:19:59.440815 21408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:19:59.457059 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:19:59.428562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35099 (local address 127.17.16.126:41483)
0811 09:19:59.428751 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:19:59.428757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:19:59.439230 (+ 10473us) server_negotiation.cc:407] Connection header received
0811 09:19:59.439768 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:19:59.439773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:19:59.439859 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:19:59.439933 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:19:59.441764 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:59.442936 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:59.445007 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:19:59.445682 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:19:59.445805 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:19:59.449218 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:19:59.449225 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:19:59.449230 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:19:59.449271 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:19:59.452946 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:59.453945 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:59.453950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:59.453953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:59.454092 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:19:59.454678 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:19:59.454684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:19:59.454687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:19:59.455115 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:19:59.455390 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 09:19:59.456487 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0811 09:19:59.456664 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:19:59.457234 21408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:19:59.467479 21408 catalog_manager.cc:1347] Generated new cluster ID: 5b37746e1353472aaaff6dd23c2dcc57
I20260811 09:19:59.467602 21408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:19:59.476600 21408 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:19:59.478634 21408 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:19:59.487542 21408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94fb5ec76f084aa283b127a803b26f5d: Generated new TSK 0
I20260811 09:19:59.488642 21408 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:19:59.500538 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21255
I20260811 09:19:59.502442 21370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:19:59.592978 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 21255
W20260811 09:19:59.614465 21230 connection.cc:535] client connection to 127.17.16.126:41483 recv error: Network error: failed to read from TLS socket (remote: 127.17.16.126:41483): Connection reset by peer (error 104)
2026-08-11T09:19:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2259 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:19:59 dist-test-slave-ncp9 krb5kdc[21450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:19:59 dist-test-slave-ncp9 krb5kdc[21450](info): set up 2 sockets
Aug 11 09:19:59 dist-test-slave-ncp9 krb5kdc[21450](info): commencing operation
krb5kdc: starting...
W20260811 09:20:00.836364 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.005s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:20:01 dist-test-slave-ncp9 krb5kdc[21450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:20:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:20:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:20:01.159657 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:34915
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:46767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:34915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:20:01.501562 21477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:01.501863 21477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:01.502005 21477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:01.509310 21477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:01.509439 21477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:01.509531 21477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:01.509610 21477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:01.509671 21477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:01.519598 21477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:46767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:34915
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:34915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:01.520877 21477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:01.522240 21477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:20:01.533221 21485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:01.533273 21486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:01.533299 21490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:01.534160 21477 server_base.cc:1034] running on GCE node
I20260811 09:20:01.534878 21477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:01.536151 21477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:01.537359 21477 hybrid_clock.cc:648] HybridClock initialized: now 1786440001537309 us; error 38 us; skew 500 ppm
I20260811 09:20:01.537712 21477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:01 dist-test-slave-ncp9 krb5kdc[21450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440001, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:01.540612 21477 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:20:01.542034 21477 webserver.cc:460] Webserver started at http://127.17.16.126:46555/ using document root <none> and password file <none>
I20260811 09:20:01.542699 21477 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:01.542814 21477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:01.543097 21477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:01.546083 21477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "b59b2f149b36469d993aa920bc28473a"
format_stamp: "Formatted at 2026-08-11 09:20:01 on dist-test-slave-ncp9"
server_key: "9be480bee1a2cf5d6c3b1fdd2c551fe9"
server_key_iv: "86ab5cb60a1cd62262067459ab5abdc1"
server_key_version: "encryptionkey@0"
I20260811 09:20:01.546831 21477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "b59b2f149b36469d993aa920bc28473a"
format_stamp: "Formatted at 2026-08-11 09:20:01 on dist-test-slave-ncp9"
server_key: "9be480bee1a2cf5d6c3b1fdd2c551fe9"
server_key_iv: "86ab5cb60a1cd62262067459ab5abdc1"
server_key_version: "encryptionkey@0"
I20260811 09:20:01.551807 21477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 09:20:01.555087 21500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:01.556334 21477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:20:01.556479 21477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "b59b2f149b36469d993aa920bc28473a"
format_stamp: "Formatted at 2026-08-11 09:20:01 on dist-test-slave-ncp9"
server_key: "9be480bee1a2cf5d6c3b1fdd2c551fe9"
server_key_iv: "86ab5cb60a1cd62262067459ab5abdc1"
server_key_version: "encryptionkey@0"
I20260811 09:20:01.556671 21477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:01.571239 21477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:01.575069 21477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:01.575328 21477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:01.594568 21477 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:34915
I20260811 09:20:01.594590 21587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:34915 every 8 connection(s)
I20260811 09:20:01.595919 21477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:20:01.599571 21588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:01.600256 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 21477
I20260811 09:20:01.600448 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:20:01.601063 17473 external_mini_cluster.cc:1428] Setting key b1ceaa94cb88e577461135f7067f35c3
I20260811 09:20:01.612912 21588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a: Bootstrap starting.
I20260811 09:20:01.614360 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:01.602862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35331 (local address 127.17.16.126:34915)
0811 09:20:01.603598 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:01.603612 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:01.603654 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:20:01.605723 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:01.605747 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:01.606149 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:01.606464 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:01.607040 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:01.608261 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:01.610182 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:01.610850 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:01.610972 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:01.612193 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:01.612200 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:01.612213 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:20:01.612711 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:01.612783 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:20:01.612807 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:01.612963 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:01.613215 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 09:20:01.613381 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":327,"threads_started":1}
I20260811 09:20:01.616308 21588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:01.617534 21588 log.cc:824] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:01.617933 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:20:01.620043 21588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a: No bootstrap required, opened a new log
I20260811 09:20:01.628571 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:01.622100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60759 (local address 127.17.16.126:34915)
0811 09:20:01.622290 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:01.622296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:01.622827 (+ 531us) server_negotiation.cc:407] Connection header received
0811 09:20:01.622974 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:01.622977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:01.623044 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:01.623121 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:01.623660 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:01.624434 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:01.626220 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:01.626669 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:01.626753 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:01.627890 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:01.627894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:01.627897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:20:01.627913 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:01.627967 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:20:01.627985 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:01.628076 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:01.628231 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 09:20:01.628273 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 09:20:01.629315 21588 raft_consensus.cc:348] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b2f149b36469d993aa920bc28473a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 34915 } }
I20260811 09:20:01.629669 21588 raft_consensus.cc:374] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:20:01.629777 21588 raft_consensus.cc:729] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b59b2f149b36469d993aa920bc28473a, State: Initialized, Role: FOLLOWER
I20260811 09:20:01.630365 21588 consensus_queue.cc:260] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b2f149b36469d993aa920bc28473a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 34915 } }
I20260811 09:20:01.630586 21588 raft_consensus.cc:388] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:20:01.630713 21588 raft_consensus.cc:482] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:20:01.630857 21588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:01.633001 21588 raft_consensus.cc:504] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b2f149b36469d993aa920bc28473a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 34915 } }
I20260811 09:20:01.633493 21588 leader_election.cc:302] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b59b2f149b36469d993aa920bc28473a; no voters:
I20260811 09:20:01.634008 21588 leader_election.cc:288] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:20:01.634140 21602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:20:01.636667 21602 raft_consensus.cc:686] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [term 1 LEADER]: Becoming Leader. State: Replica: b59b2f149b36469d993aa920bc28473a, State: Running, Role: LEADER
I20260811 09:20:01.637218 21602 consensus_queue.cc:237] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b2f149b36469d993aa920bc28473a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 34915 } }
I20260811 09:20:01.637755 21588 sys_catalog.cc:564] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:20:01.640002 21604 sys_catalog.cc:455] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b59b2f149b36469d993aa920bc28473a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b2f149b36469d993aa920bc28473a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 34915 } } }
I20260811 09:20:01.640380 21604 sys_catalog.cc:458] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:01.640735 21605 sys_catalog.cc:455] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b59b2f149b36469d993aa920bc28473a. Latest consensus state: current_term: 1 leader_uuid: "b59b2f149b36469d993aa920bc28473a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b59b2f149b36469d993aa920bc28473a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 34915 } } }
I20260811 09:20:01.641568 21605 sys_catalog.cc:458] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:01.649534 21611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:20:01.653764 21611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:20:01.661437 21611 catalog_manager.cc:1347] Generated new cluster ID: 4a156766cb274023beaf2cf46ce1187c
I20260811 09:20:01.661569 21611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:20:01.720263 21611 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:20:01.721787 21611 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:20:01.731791 21611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b59b2f149b36469d993aa920bc28473a: Generated new TSK 0
I20260811 09:20:01.732640 21611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:20:01.739049 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21477
W20260811 09:20:01.740289 21583 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35331
W20260811 09:20:01.741115 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21477: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:20:01.741258 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 21477
W20260811 09:20:01.767292 21446 connection.cc:535] client connection to 127.17.16.126:34915 recv error: Network error: failed to read from TLS socket (remote: 127.17.16.126:34915): Connection reset by peer (error 104)
2026-08-11T09:20:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2153 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:20:01 dist-test-slave-ncp9 krb5kdc[21635](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:20:01 dist-test-slave-ncp9 krb5kdc[21635](info): set up 2 sockets
Aug 11 09:20:01 dist-test-slave-ncp9 krb5kdc[21635](info): commencing operation
krb5kdc: starting...
W20260811 09:20:02.994539 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:20:03 dist-test-slave-ncp9 krb5kdc[21635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:20:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:20:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:20:03.315971 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:44259
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:44259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:20:03.657461 21658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:03.657714 21658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:03.657841 21658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:03.664757 21658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:03.664855 21658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:03.664950 21658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:03.665033 21658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:03.665089 21658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:03.674511 21658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:44259
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:44259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:03.675683 21658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:03.676999 21658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:20:03.688352 21666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:03.688318 21671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:03.688398 21668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:03.689389 21658 server_base.cc:1034] running on GCE node
I20260811 09:20:03.690127 21658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:03.691517 21658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:03.692720 21658 hybrid_clock.cc:648] HybridClock initialized: now 1786440003692664 us; error 42 us; skew 500 ppm
I20260811 09:20:03.693094 21658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:03 dist-test-slave-ncp9 krb5kdc[21635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440003, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:03.696161 21658 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:20:03.697614 21658 webserver.cc:460] Webserver started at http://127.17.16.126:38459/ using document root <none> and password file <none>
I20260811 09:20:03.698287 21658 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:03.698400 21658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:03.698691 21658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:03.701781 21658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "adb79c45b65f440a9f9a47d2a3859c23"
format_stamp: "Formatted at 2026-08-11 09:20:03 on dist-test-slave-ncp9"
server_key: "3e8120148767d2cdc69746a0ad243044"
server_key_iv: "7af8773d08b08ff5e96ee7d7c4030b9d"
server_key_version: "encryptionkey@0"
I20260811 09:20:03.702577 21658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "adb79c45b65f440a9f9a47d2a3859c23"
format_stamp: "Formatted at 2026-08-11 09:20:03 on dist-test-slave-ncp9"
server_key: "3e8120148767d2cdc69746a0ad243044"
server_key_iv: "7af8773d08b08ff5e96ee7d7c4030b9d"
server_key_version: "encryptionkey@0"
I20260811 09:20:03.707705 21658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:20:03.711069 21681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:03.712313 21658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:20:03.712468 21658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "adb79c45b65f440a9f9a47d2a3859c23"
format_stamp: "Formatted at 2026-08-11 09:20:03 on dist-test-slave-ncp9"
server_key: "3e8120148767d2cdc69746a0ad243044"
server_key_iv: "7af8773d08b08ff5e96ee7d7c4030b9d"
server_key_version: "encryptionkey@0"
I20260811 09:20:03.712648 21658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:03.718288 21658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:03.722139 21658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:03.722427 21658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:03.740324 21658 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:44259
I20260811 09:20:03.740351 21786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:44259 every 8 connection(s)
I20260811 09:20:03.741797 21658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:20:03.745705 21788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:03.747972 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 21658
I20260811 09:20:03.748154 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:20:03.748592 17473 external_mini_cluster.cc:1428] Setting key 14ab0a3ead4df8e7ecbd6c8a870e1a6e
I20260811 09:20:03.756364 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:03.750541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41959 (local address 127.17.16.126:44259)
0811 09:20:03.751346 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:03.751358 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:03.751402 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:20:03.753089 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:03.753111 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:03.753440 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:03.753749 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:03.754820 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:03.754826 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:03.754839 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:20:03.754864 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:03.754933 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:20:03.754956 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:03.755152 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:03.755255 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 09:20:03.755439 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":335,"threads_started":1}
I20260811 09:20:03.757728 21788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23: Bootstrap starting.
I20260811 09:20:03.759882 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:20:03.761018 21788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:03.762270 21788 log.cc:824] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:03.765230 21788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23: No bootstrap required, opened a new log
I20260811 09:20:03.767282 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:03.765015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46985 (local address 127.17.16.126:44259)
0811 09:20:03.765181 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:03.765186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:03.765497 (+ 311us) server_negotiation.cc:407] Connection header received
0811 09:20:03.765646 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:03.765651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:03.765717 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:03.765825 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:03.766652 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:03.766657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:03.766662 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:20:03.766680 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:03.766725 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:20:03.766746 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:03.766862 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:03.766913 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 09:20:03.766971 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:20:03.773474 21788 raft_consensus.cc:348] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb79c45b65f440a9f9a47d2a3859c23" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44259 } }
I20260811 09:20:03.773838 21788 raft_consensus.cc:374] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:20:03.773914 21788 raft_consensus.cc:729] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adb79c45b65f440a9f9a47d2a3859c23, State: Initialized, Role: FOLLOWER
I20260811 09:20:03.774420 21788 consensus_queue.cc:260] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb79c45b65f440a9f9a47d2a3859c23" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44259 } }
I20260811 09:20:03.774634 21788 raft_consensus.cc:388] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:20:03.774736 21788 raft_consensus.cc:482] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:20:03.774852 21788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:03.777145 21788 raft_consensus.cc:504] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb79c45b65f440a9f9a47d2a3859c23" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44259 } }
I20260811 09:20:03.777622 21788 leader_election.cc:302] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adb79c45b65f440a9f9a47d2a3859c23; no voters:
I20260811 09:20:03.778179 21788 leader_election.cc:288] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:20:03.778318 21806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:20:03.778978 21806 raft_consensus.cc:686] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [term 1 LEADER]: Becoming Leader. State: Replica: adb79c45b65f440a9f9a47d2a3859c23, State: Running, Role: LEADER
I20260811 09:20:03.779553 21806 consensus_queue.cc:237] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb79c45b65f440a9f9a47d2a3859c23" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44259 } }
I20260811 09:20:03.780352 21788 sys_catalog.cc:564] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:20:03.782335 21811 sys_catalog.cc:455] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adb79c45b65f440a9f9a47d2a3859c23. Latest consensus state: current_term: 1 leader_uuid: "adb79c45b65f440a9f9a47d2a3859c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb79c45b65f440a9f9a47d2a3859c23" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44259 } } }
I20260811 09:20:03.782770 21811 sys_catalog.cc:458] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:03.784313 21816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:20:03.785061 21809 sys_catalog.cc:455] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adb79c45b65f440a9f9a47d2a3859c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adb79c45b65f440a9f9a47d2a3859c23" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 44259 } } }
I20260811 09:20:03.785359 21809 sys_catalog.cc:458] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23 [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:03.789451 21816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:20:03.798727 21816 catalog_manager.cc:1347] Generated new cluster ID: 38308e7dcf4f44f3a1b42d7eb5ab8f94
I20260811 09:20:03.798830 21816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:20:03.809721 21816 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:20:03.810165 21816 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:20:03.816856 21816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adb79c45b65f440a9f9a47d2a3859c23: Generated new TSK 0
I20260811 09:20:03.817584 21816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:20:03.828984 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21658
W20260811 09:20:03.830041 21779 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41959
W20260811 09:20:03.830727 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21658: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:20:03.830888 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 21658
2026-08-11T09:20:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2088 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:20:03 dist-test-slave-ncp9 krb5kdc[21852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:20:03 dist-test-slave-ncp9 krb5kdc[21852](info): set up 2 sockets
Aug 11 09:20:03 dist-test-slave-ncp9 krb5kdc[21852](info): commencing operation
krb5kdc: starting...
W20260811 09:20:05.090372 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:20:05 dist-test-slave-ncp9 krb5kdc[21852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:20:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:20:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:20:05.416477 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:43433
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:43433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:20:05.755916 21874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:05.756210 21874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:05.756345 21874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:05.763509 21874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:05.763629 21874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:05.763687 21874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:05.763775 21874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:05.763828 21874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:05.773545 21874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:43433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:43433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:05.774816 21874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:05.776113 21874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:20:05.787549 21888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:05.787649 21892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:05.787621 21889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:05.788743 21874 server_base.cc:1034] running on GCE node
I20260811 09:20:05.789530 21874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:05.790905 21874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:05.792110 21874 hybrid_clock.cc:648] HybridClock initialized: now 1786440005792074 us; error 49 us; skew 500 ppm
I20260811 09:20:05.792449 21874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:05 dist-test-slave-ncp9 krb5kdc[21852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440005, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:05.798309 21874 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:20:05.799736 21874 webserver.cc:460] Webserver started at http://127.17.16.126:43341/ using document root <none> and password file <none>
I20260811 09:20:05.800414 21874 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:05.800535 21874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:05.800827 21874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:05.803835 21874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "08fb79cc96c3461aac0416e8177fe5cf"
format_stamp: "Formatted at 2026-08-11 09:20:05 on dist-test-slave-ncp9"
server_key: "023213b611e79f7f537769e9a17e5d90"
server_key_iv: "e4430e32420382de38a4467cb1d1be73"
server_key_version: "encryptionkey@0"
I20260811 09:20:05.804580 21874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "08fb79cc96c3461aac0416e8177fe5cf"
format_stamp: "Formatted at 2026-08-11 09:20:05 on dist-test-slave-ncp9"
server_key: "023213b611e79f7f537769e9a17e5d90"
server_key_iv: "e4430e32420382de38a4467cb1d1be73"
server_key_version: "encryptionkey@0"
I20260811 09:20:05.809631 21874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:20:05.812884 21910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:05.814177 21874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:20:05.814338 21874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "08fb79cc96c3461aac0416e8177fe5cf"
format_stamp: "Formatted at 2026-08-11 09:20:05 on dist-test-slave-ncp9"
server_key: "023213b611e79f7f537769e9a17e5d90"
server_key_iv: "e4430e32420382de38a4467cb1d1be73"
server_key_version: "encryptionkey@0"
I20260811 09:20:05.814509 21874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:05.829893 21874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:05.833721 21874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:05.833989 21874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:05.851245 21874 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:43433
I20260811 09:20:05.851276 22002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:43433 every 8 connection(s)
I20260811 09:20:05.852499 21874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:20:05.856093 22004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:05.857932 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 21874
I20260811 09:20:05.858106 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:20:05.858613 17473 external_mini_cluster.cc:1428] Setting key 2818399c3bcdb555795d43c38b5477ba
I20260811 09:20:05.869520 22004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf: Bootstrap starting.
Aug 11 09:20:05 dist-test-slave-ncp9 krb5kdc[21852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:20:05.873368 22004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:05.874627 22004 log.cc:824] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:05.877669 22004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf: No bootstrap required, opened a new log
I20260811 09:20:05.881855 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:05.860267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33889 (local address 127.17.16.126:43433)
0811 09:20:05.861070 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:05.861081 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:05.861121 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:20:05.862884 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:05.862908 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:05.863322 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:05.863680 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:05.865580 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:05.866942 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:05.868695 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:05.869418 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:05.869570 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:05.874468 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:05.874476 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:05.874490 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:05.874554 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:05.877714 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:05.878429 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:05.878436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:05.878442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:05.878626 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:05.879016 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:05.879020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:05.879022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:05.879610 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:05.879913 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:05.880380 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 09:20:05.880788 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":364,"threads_started":1}
I20260811 09:20:05.885855 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:20:05.886080 22004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08fb79cc96c3461aac0416e8177fe5cf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43433 } }
I20260811 09:20:05.886408 22004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:20:05.886505 22004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08fb79cc96c3461aac0416e8177fe5cf, State: Initialized, Role: FOLLOWER
I20260811 09:20:05.887079 22004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08fb79cc96c3461aac0416e8177fe5cf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43433 } }
I20260811 09:20:05.887308 22004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:20:05.887432 22004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:20:05.887586 22004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:05.889902 22004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08fb79cc96c3461aac0416e8177fe5cf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43433 } }
I20260811 09:20:05.890456 22004 leader_election.cc:302] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08fb79cc96c3461aac0416e8177fe5cf; no voters:
I20260811 09:20:05.891053 22004 leader_election.cc:288] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:20:05.891167 22023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:20:05.892992 22004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:20:05.893484 22023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [term 1 LEADER]: Becoming Leader. State: Replica: 08fb79cc96c3461aac0416e8177fe5cf, State: Running, Role: LEADER
I20260811 09:20:05.894032 22023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08fb79cc96c3461aac0416e8177fe5cf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43433 } }
I20260811 09:20:05.896611 22025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08fb79cc96c3461aac0416e8177fe5cf. Latest consensus state: current_term: 1 leader_uuid: "08fb79cc96c3461aac0416e8177fe5cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08fb79cc96c3461aac0416e8177fe5cf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43433 } } }
I20260811 09:20:05.897022 22025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:05.898151 22024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08fb79cc96c3461aac0416e8177fe5cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08fb79cc96c3461aac0416e8177fe5cf" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43433 } } }
I20260811 09:20:05.898448 22024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:05.900839 22032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:20:05.905722 22032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:20:05.908799 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:05.890203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46197 (local address 127.17.16.126:43433)
0811 09:20:05.890377 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:05.890383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:05.890836 (+ 453us) server_negotiation.cc:407] Connection header received
0811 09:20:05.890998 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:05.891006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:05.891095 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:05.891216 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:05.893233 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:05.894372 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:05.896317 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:05.897153 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:05.897287 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:05.900574 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:05.900581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:05.900586 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:05.900625 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:05.904592 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:05.905514 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:05.905521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:05.905528 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:05.905714 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:05.906174 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:05.906179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:05.906183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:05.906668 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:05.906940 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:05.908115 (+ 1175us) server_negotiation.cc:299] Negotiation successful
0811 09:20:05.908376 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:20:05.917009 22032 catalog_manager.cc:1347] Generated new cluster ID: c339b34fc73e49aea0785086a1e269a9
I20260811 09:20:05.917105 22032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:20:05.935935 22032 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:20:05.937248 22032 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:20:05.944214 22032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08fb79cc96c3461aac0416e8177fe5cf: Generated new TSK 0
I20260811 09:20:05.944989 22032 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:20:05.950991 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 21874
I20260811 09:20:05.951959 21995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:20:06.040710 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 21874
2026-08-11T09:20:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2206 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:20:06 dist-test-slave-ncp9 krb5kdc[22070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:20:06 dist-test-slave-ncp9 krb5kdc[22070](info): set up 2 sockets
Aug 11 09:20:06 dist-test-slave-ncp9 krb5kdc[22070](info): commencing operation
krb5kdc: starting...
W20260811 09:20:07.288479 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:20:07 dist-test-slave-ncp9 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:20:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:20:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:20:07.613667 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:43197
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:43197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:20:07.954928 22096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:07.955201 22096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:07.955300 22096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:07.962337 22096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:07.962441 22096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:07.962493 22096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:07.962553 22096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:07.962596 22096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:07.972110 22096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:43197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:43197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:07.973346 22096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:07.974608 22096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:20:07.986303 22122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:07.986332 22117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:07.986588 22105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:07.986864 22096 server_base.cc:1034] running on GCE node
I20260811 09:20:07.987742 22096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:07.989115 22096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:07.990321 22096 hybrid_clock.cc:648] HybridClock initialized: now 1786440007990270 us; error 42 us; skew 500 ppm
I20260811 09:20:07.990650 22096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:07 dist-test-slave-ncp9 krb5kdc[22070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440007, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:07.993635 22096 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:20:07.994951 22096 webserver.cc:460] Webserver started at http://127.17.16.126:33899/ using document root <none> and password file <none>
I20260811 09:20:07.995584 22096 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:07.995693 22096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:07.995976 22096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:07.998950 22096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "a51b205ebbfb4132a1c7b56c0f57435a"
format_stamp: "Formatted at 2026-08-11 09:20:07 on dist-test-slave-ncp9"
server_key: "35b20425168da604ccefef6a788ed4d9"
server_key_iv: "391e7cf1cfaeb9633052ed66b2ddccee"
server_key_version: "encryptionkey@0"
I20260811 09:20:07.999675 22096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "a51b205ebbfb4132a1c7b56c0f57435a"
format_stamp: "Formatted at 2026-08-11 09:20:07 on dist-test-slave-ncp9"
server_key: "35b20425168da604ccefef6a788ed4d9"
server_key_iv: "391e7cf1cfaeb9633052ed66b2ddccee"
server_key_version: "encryptionkey@0"
I20260811 09:20:08.004642 22096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 09:20:08.007903 22135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:08.009158 22096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:20:08.009305 22096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "a51b205ebbfb4132a1c7b56c0f57435a"
format_stamp: "Formatted at 2026-08-11 09:20:07 on dist-test-slave-ncp9"
server_key: "35b20425168da604ccefef6a788ed4d9"
server_key_iv: "391e7cf1cfaeb9633052ed66b2ddccee"
server_key_version: "encryptionkey@0"
I20260811 09:20:08.009469 22096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:08.041680 22096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:08.045478 22096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:08.045744 22096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:08.063344 22096 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:43197
I20260811 09:20:08.063369 22218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:43197 every 8 connection(s)
I20260811 09:20:08.064694 22096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:20:08.066228 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 22096
I20260811 09:20:08.066380 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:20:08.066841 17473 external_mini_cluster.cc:1428] Setting key 1f982e0f3ca78c2ee6c5c54052a4fef3
I20260811 09:20:08.068794 22222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:08.079782 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:08.068575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34775 (local address 127.17.16.126:43197)
0811 09:20:08.069441 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:08.069453 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:08.069494 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:20:08.071198 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:08.071220 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:08.071597 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:08.071913 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:08.072499 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:08.073863 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:08.075609 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:08.076233 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:08.076361 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:08.077598 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:08.077605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:08.077618 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:20:08.078148 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:08.078233 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:20:08.078261 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:08.078349 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:08.078569 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 09:20:08.078804 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":388,"threads_started":1}
I20260811 09:20:08.081013 22222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a: Bootstrap starting.
I20260811 09:20:08.083186 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:20:08.084239 22222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:08.085479 22222 log.cc:824] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:08.089010 22222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a: No bootstrap required, opened a new log
I20260811 09:20:08.095361 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:08.088795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40409 (local address 127.17.16.126:43197)
0811 09:20:08.088990 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:08.088999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:08.089535 (+ 536us) server_negotiation.cc:407] Connection header received
0811 09:20:08.089811 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:08.089815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:08.089876 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:08.089955 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:08.090557 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:08.091290 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:08.093111 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:08.093537 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:08.093625 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:08.094694 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:08.094699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:08.094702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:20:08.094720 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:08.094760 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:20:08.094775 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:08.094846 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:08.094979 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 09:20:08.095052 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:20:08.098383 22222 raft_consensus.cc:348] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b205ebbfb4132a1c7b56c0f57435a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43197 } }
I20260811 09:20:08.098739 22222 raft_consensus.cc:374] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:20:08.098830 22222 raft_consensus.cc:729] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a51b205ebbfb4132a1c7b56c0f57435a, State: Initialized, Role: FOLLOWER
I20260811 09:20:08.099390 22222 consensus_queue.cc:260] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b205ebbfb4132a1c7b56c0f57435a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43197 } }
I20260811 09:20:08.099609 22222 raft_consensus.cc:388] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:20:08.099741 22222 raft_consensus.cc:482] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:20:08.099892 22222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:08.102428 22222 raft_consensus.cc:504] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b205ebbfb4132a1c7b56c0f57435a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43197 } }
I20260811 09:20:08.103080 22222 leader_election.cc:302] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a51b205ebbfb4132a1c7b56c0f57435a; no voters:
I20260811 09:20:08.103717 22222 leader_election.cc:288] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:20:08.103806 22241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:20:08.104494 22241 raft_consensus.cc:686] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [term 1 LEADER]: Becoming Leader. State: Replica: a51b205ebbfb4132a1c7b56c0f57435a, State: Running, Role: LEADER
I20260811 09:20:08.105075 22241 consensus_queue.cc:237] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b205ebbfb4132a1c7b56c0f57435a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43197 } }
I20260811 09:20:08.105860 22222 sys_catalog.cc:564] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:20:08.107803 22242 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a51b205ebbfb4132a1c7b56c0f57435a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b205ebbfb4132a1c7b56c0f57435a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43197 } } }
I20260811 09:20:08.108248 22242 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:08.109215 22244 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a51b205ebbfb4132a1c7b56c0f57435a. Latest consensus state: current_term: 1 leader_uuid: "a51b205ebbfb4132a1c7b56c0f57435a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51b205ebbfb4132a1c7b56c0f57435a" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 43197 } } }
I20260811 09:20:08.109686 22244 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:08.110100 22251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:20:08.115720 22251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:20:08.125515 22251 catalog_manager.cc:1347] Generated new cluster ID: 4abb4c224c284a49a3871e562d21235b
I20260811 09:20:08.125626 22251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:20:08.147698 22251 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:20:08.149556 22251 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:20:08.160034 22251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a51b205ebbfb4132a1c7b56c0f57435a: Generated new TSK 0
I20260811 09:20:08.161178 22251 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:20:08.176774 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22096
W20260811 09:20:08.178285 22210 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34775
W20260811 09:20:08.179001 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22096: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:20:08.179147 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 22096
2026-08-11T09:20:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2140 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:20:08 dist-test-slave-ncp9 krb5kdc[22278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:20:08 dist-test-slave-ncp9 krb5kdc[22278](info): set up 2 sockets
Aug 11 09:20:08 dist-test-slave-ncp9 krb5kdc[22278](info): commencing operation
krb5kdc: starting...
W20260811 09:20:09.435496 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.009s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:20:09 dist-test-slave-ncp9 krb5kdc[22278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:20:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:20:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:20:09.771970 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:45705
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:41443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:45705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:20:10.113238 22299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:10.113531 22299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:10.113636 22299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:10.120934 22299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:10.121048 22299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:10.121101 22299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:10.121168 22299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:10.121212 22299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:10.131084 22299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:41443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:45705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:45705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:10.132295 22299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:10.133611 22299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:20:10.144855 22309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:10.144816 22313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:10.144912 22311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:10.145681 22299 server_base.cc:1034] running on GCE node
I20260811 09:20:10.146466 22299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:10.147815 22299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:10.148976 22299 hybrid_clock.cc:648] HybridClock initialized: now 1786440010148930 us; error 40 us; skew 500 ppm
I20260811 09:20:10.149305 22299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:10 dist-test-slave-ncp9 krb5kdc[22278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440010, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:10.152264 22299 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:20:10.153715 22299 webserver.cc:460] Webserver started at http://127.17.16.126:46597/ using document root <none> and password file <none>
I20260811 09:20:10.154404 22299 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:10.154522 22299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:10.154824 22299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:10.157836 22299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "56f876376be34d208c44110d6bcae8ff"
format_stamp: "Formatted at 2026-08-11 09:20:10 on dist-test-slave-ncp9"
server_key: "0a4c9d94a1ac7250962f690c1ef6355c"
server_key_iv: "ba7abb6671d8d6bc5bdc0c32dfdf7359"
server_key_version: "encryptionkey@0"
I20260811 09:20:10.158581 22299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "56f876376be34d208c44110d6bcae8ff"
format_stamp: "Formatted at 2026-08-11 09:20:10 on dist-test-slave-ncp9"
server_key: "0a4c9d94a1ac7250962f690c1ef6355c"
server_key_iv: "ba7abb6671d8d6bc5bdc0c32dfdf7359"
server_key_version: "encryptionkey@0"
I20260811 09:20:10.163673 22299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 09:20:10.167121 22327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:10.168406 22299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:20:10.168565 22299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "56f876376be34d208c44110d6bcae8ff"
format_stamp: "Formatted at 2026-08-11 09:20:10 on dist-test-slave-ncp9"
server_key: "0a4c9d94a1ac7250962f690c1ef6355c"
server_key_iv: "ba7abb6671d8d6bc5bdc0c32dfdf7359"
server_key_version: "encryptionkey@0"
I20260811 09:20:10.168756 22299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:10.174293 22299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:10.178074 22299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:10.178345 22299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:10.194657 22299 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:45705
I20260811 09:20:10.194686 22422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:45705 every 8 connection(s)
I20260811 09:20:10.195990 22299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:20:10.199745 22423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:10.204763 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 22299
I20260811 09:20:10.204955 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:20:10.205386 17473 external_mini_cluster.cc:1428] Setting key 2066b7be8b86587abc05432634dc1f76
I20260811 09:20:10.210429 22423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff: Bootstrap starting.
I20260811 09:20:10.213056 22427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:10.207164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46977 (local address 127.17.16.126:45705)
0811 09:20:10.208015 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:10.208029 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:10.208073 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:20:10.209807 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:10.209831 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:10.210176 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:10.210481 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:10.211517 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:10.211523 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:10.211535 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:20:10.211548 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:10.211608 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:20:10.211627 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:10.211809 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:10.211907 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 09:20:10.212081 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":397,"threads_started":1}
I20260811 09:20:10.213953 22423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:10.215292 22423 log.cc:824] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:10.216995 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:20:10.218286 22423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff: No bootstrap required, opened a new log
I20260811 09:20:10.223446 22427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:10.221053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51411 (local address 127.17.16.126:45705)
0811 09:20:10.221251 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:10.221261 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:10.221729 (+ 468us) server_negotiation.cc:407] Connection header received
0811 09:20:10.221851 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:10.221857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:10.221914 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:10.222034 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:10.222831 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:10.222837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:10.222841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:20:10.222863 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:10.222913 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:20:10.222932 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:10.223048 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:10.223094 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 09:20:10.223160 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:20:10.226578 22423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f876376be34d208c44110d6bcae8ff" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45705 } }
I20260811 09:20:10.226925 22423 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:20:10.226999 22423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f876376be34d208c44110d6bcae8ff, State: Initialized, Role: FOLLOWER
I20260811 09:20:10.227483 22423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f876376be34d208c44110d6bcae8ff" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45705 } }
I20260811 09:20:10.227702 22423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:20:10.227797 22423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:20:10.227928 22423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:10.230423 22423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f876376be34d208c44110d6bcae8ff" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45705 } }
I20260811 09:20:10.230898 22423 leader_election.cc:302] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56f876376be34d208c44110d6bcae8ff; no voters:
I20260811 09:20:10.231403 22423 leader_election.cc:288] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:20:10.231528 22440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:20:10.232120 22440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [term 1 LEADER]: Becoming Leader. State: Replica: 56f876376be34d208c44110d6bcae8ff, State: Running, Role: LEADER
I20260811 09:20:10.232659 22440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f876376be34d208c44110d6bcae8ff" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45705 } }
I20260811 09:20:10.233453 22423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:20:10.235762 22441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56f876376be34d208c44110d6bcae8ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f876376be34d208c44110d6bcae8ff" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45705 } } }
I20260811 09:20:10.236200 22444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56f876376be34d208c44110d6bcae8ff. Latest consensus state: current_term: 1 leader_uuid: "56f876376be34d208c44110d6bcae8ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56f876376be34d208c44110d6bcae8ff" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 45705 } } }
I20260811 09:20:10.236486 22441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:10.236595 22444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:10.238175 22458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:20:10.243089 22458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:20:10.252651 22458 catalog_manager.cc:1347] Generated new cluster ID: 03453e8741864ddf803d8addb4e69be6
I20260811 09:20:10.252780 22458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:20:10.271351 22458 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:20:10.271893 22458 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:20:10.279754 22458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56f876376be34d208c44110d6bcae8ff: Generated new TSK 0
I20260811 09:20:10.280809 22458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:20:10.288106 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22299
W20260811 09:20:10.289214 22415 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46977
W20260811 09:20:10.289928 17473 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22299: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:20:10.290091 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 22299
2026-08-11T09:20:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2117 ms)
[----------] 18 tests from AuthTokenIssuingTest (38947 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:20:10 dist-test-slave-ncp9 krb5kdc[22475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:20:10 dist-test-slave-ncp9 krb5kdc[22475](info): set up 2 sockets
Aug 11 09:20:10 dist-test-slave-ncp9 krb5kdc[22475](info): commencing operation
krb5kdc: starting...
W20260811 09:20:11.555168 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:20:11 dist-test-slave-ncp9 krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:20:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:20:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:20:11.883464 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41019
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41019,127.17.16.125:35225,127.17.16.124:45223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:20:12.219307 22502 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:20:12.219659 22502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:12.219758 22502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:12.219875 22502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:12.227067 22502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:12.227169 22502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:12.227253 22502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:12.227319 22502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:12.227406 22502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:12.237017 22502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41019,127.17.16.125:35225,127.17.16.124:45223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:41019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:12.238327 22502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:12.239624 22502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:20:12.251107 22513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:12.251036 22526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:12.251083 22514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:12.251792 22502 server_base.cc:1034] running on GCE node
I20260811 09:20:12.252563 22502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:12.253945 22502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:12.255153 22502 hybrid_clock.cc:648] HybridClock initialized: now 1786440012255085 us; error 47 us; skew 500 ppm
I20260811 09:20:12.255441 22502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:12 dist-test-slave-ncp9 krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:12.261845 22502 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:20:12.263191 22502 webserver.cc:460] Webserver started at http://127.17.16.126:46597/ using document root <none> and password file <none>
I20260811 09:20:12.263813 22502 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:12.263896 22502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:12.264163 22502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:12.267103 22502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "c43e3efb1ae94edda2cb932a425ae40c"
format_stamp: "Formatted at 2026-08-11 09:20:12 on dist-test-slave-ncp9"
server_key: "ec8a52af3df5bc53a85c38b5a8457190"
server_key_iv: "002141daf01e167bc99b8e4740b4e9ed"
server_key_version: "encryptionkey@0"
I20260811 09:20:12.267834 22502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "c43e3efb1ae94edda2cb932a425ae40c"
format_stamp: "Formatted at 2026-08-11 09:20:12 on dist-test-slave-ncp9"
server_key: "ec8a52af3df5bc53a85c38b5a8457190"
server_key_iv: "002141daf01e167bc99b8e4740b4e9ed"
server_key_version: "encryptionkey@0"
I20260811 09:20:12.272702 22502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 09:20:12.275943 22537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:12.277482 22502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 09:20:12.277650 22502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "c43e3efb1ae94edda2cb932a425ae40c"
format_stamp: "Formatted at 2026-08-11 09:20:12 on dist-test-slave-ncp9"
server_key: "ec8a52af3df5bc53a85c38b5a8457190"
server_key_iv: "002141daf01e167bc99b8e4740b4e9ed"
server_key_version: "encryptionkey@0"
I20260811 09:20:12.277851 22502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:12.295545 22502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:12.299853 22502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:12.300114 22502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:12.317773 22502 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:41019
I20260811 09:20:12.317792 22622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:41019 every 8 connection(s)
I20260811 09:20:12.319151 22502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:20:12.322896 22623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:12.325093 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 22502
I20260811 09:20:12.325278 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:20:12.325923 17473 external_mini_cluster.cc:1428] Setting key c6a0788517df96798276129f826f5bba
I20260811 09:20:12.329576 22623 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:20:12 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:20:12.351922 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.332621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41019 (local address 127.0.0.1:45434)
0811 09:20:12.333456 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.333502 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:12.335412 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:12.336199 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:12.336207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:12.336879 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:12.337255 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:12.337273 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.338855 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.338861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:12.340349 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:12.340366 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.341411 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.341419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:12.341603 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:12.342275 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:12.342297 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:12.344422 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:12.348150 (+ 3728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:12.348161 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:12.348170 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:12.348559 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:12.349028 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:12.349031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:12.349033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:12.349185 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:12.350112 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:12.350117 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:12.350300 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 09:20:12.350502 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 09:20:12.350882 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":352,"threads_started":1}
I20260811 09:20:12.351969 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.332622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45434 (local address 127.17.16.126:41019)
0811 09:20:12.334213 (+ 1591us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:12.334220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:12.334273 (+ 53us) server_negotiation.cc:407] Connection header received
0811 09:20:12.335753 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:12.335758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:12.335977 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:12.336155 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:12.337451 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.338682 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.340497 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.341171 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.341297 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:12.344664 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:12.344674 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:12.344690 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:12.344732 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:12.348017 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:12.348703 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:12.348713 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:12.348716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:12.348922 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:12.349287 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:12.349291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:12.349294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:12.349986 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:12.350170 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:12.350661 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 09:20:12.351067 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":1375,"thread_start_us":1172,"threads_started":1}
I20260811 09:20:12.354794 22623 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:20:12.355914 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.355531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35892)
0811 09:20:12.355697 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.355805 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:20:12.356236 22542 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
I20260811 09:20:12.357820 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.357456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35896)
0811 09:20:12.357631 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.357699 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 09:20:12.358036 22623 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:35225: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.16.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab.
Entry for principal kudu/127.17.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab.
I20260811 09:20:12.398033 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.125:35225
--webserver_interface=127.17.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab
--principal=kudu/127.17.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41019,127.17.16.125:35225,127.17.16.124:45223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:20:12.407343 22623 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } attempt: 1
I20260811 09:20:12.408418 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.407968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35912)
0811 09:20:12.408177 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.408281 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 09:20:12.409447 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.409106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35928)
0811 09:20:12.409257 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.409321 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:20:12.409634 22623 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:35225: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
I20260811 09:20:12.477905 22623 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } attempt: 2
I20260811 09:20:12.479243 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.478665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35938)
0811 09:20:12.478937 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.479052 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 09:20:12.480319 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.479880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35948)
0811 09:20:12.480041 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.480156 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 09:20:12.480540 22623 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:35225: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
I20260811 09:20:12.571841 22623 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } attempt: 3
I20260811 09:20:12.573258 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.572684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35950)
0811 09:20:12.572956 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.573073 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 09:20:12.574331 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.573902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35966)
0811 09:20:12.574067 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.574163 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 09:20:12.574522 22623 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:35225: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
I20260811 09:20:12.717797 22623 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } attempt: 4
I20260811 09:20:12.719165 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.718491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35972)
0811 09:20:12.718853 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.718968 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198}
I20260811 09:20:12.720239 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.719798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35976)
0811 09:20:12.719972 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.720085 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 09:20:12.720444 22623 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:35225: Network error: Client connection negotiation failed: client connection to 127.17.16.125:35225: connect: Connection refused (error 111)
W20260811 09:20:12.734267 22636 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:20:12.734584 22636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:12.734658 22636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:12.734738 22636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:12.741909 22636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:12.741995 22636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:12.742054 22636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:12.742105 22636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:12.742166 22636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:12.751964 22636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41019,127.17.16.125:35225,127.17.16.124:45223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab
--principal=kudu/127.17.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.125:35225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:12.753326 22636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:12.754631 22636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:20:12.766332 22643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:12.766352 22645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:12.766300 22647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:12.767342 22636 server_base.cc:1034] running on GCE node
I20260811 09:20:12.768087 22636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:12.769549 22636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:12.770766 22636 hybrid_clock.cc:648] HybridClock initialized: now 1786440012770694 us; error 39 us; skew 500 ppm
I20260811 09:20:12.771081 22636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:12 dist-test-slave-ncp9 krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:12.774116 22636 init.cc:377] Logged in from keytab as kudu/127.17.16.125@KRBTEST.COM (short username kudu)
I20260811 09:20:12.775511 22636 webserver.cc:460] Webserver started at http://127.17.16.125:41539/ using document root <none> and password file <none>
I20260811 09:20:12.776204 22636 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:12.776309 22636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:12.776613 22636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:12.779671 22636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/data/instance:
uuid: "454e6ab6f26345c58aa4d0c0feb11b03"
format_stamp: "Formatted at 2026-08-11 09:20:12 on dist-test-slave-ncp9"
server_key: "443313d1e67f2fea2cbc858cef9f720a"
server_key_iv: "2e34d9697ee8e273e7088fcb84ab544c"
server_key_version: "encryptionkey@0"
I20260811 09:20:12.780443 22636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/wal/instance:
uuid: "454e6ab6f26345c58aa4d0c0feb11b03"
format_stamp: "Formatted at 2026-08-11 09:20:12 on dist-test-slave-ncp9"
server_key: "443313d1e67f2fea2cbc858cef9f720a"
server_key_iv: "2e34d9697ee8e273e7088fcb84ab544c"
server_key_version: "encryptionkey@0"
I20260811 09:20:12.785753 22636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:20:12.789273 22659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:12.790627 22636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:20:12.790798 22636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/wal
uuid: "454e6ab6f26345c58aa4d0c0feb11b03"
format_stamp: "Formatted at 2026-08-11 09:20:12 on dist-test-slave-ncp9"
server_key: "443313d1e67f2fea2cbc858cef9f720a"
server_key_iv: "2e34d9697ee8e273e7088fcb84ab544c"
server_key_version: "encryptionkey@0"
I20260811 09:20:12.791014 22636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:12.816229 22636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:12.820227 22636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:12.820492 22636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:12.838019 22636 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.125:35225
I20260811 09:20:12.838038 22757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.125:35225 every 8 connection(s)
I20260811 09:20:12.839331 22636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
I20260811 09:20:12.839866 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 22636
I20260811 09:20:12.840034 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-1/wal/instance
I20260811 09:20:12.840592 17473 external_mini_cluster.cc:1428] Setting key 6e1939fbcc5505c00696afa6c5b55820
I20260811 09:20:12.843225 22758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:12.849614 22758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:20:12 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:20:12.869234 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.852102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.17.16.126:41019)
0811 09:20:12.852306 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:12.852312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:12.853523 (+ 1211us) server_negotiation.cc:407] Connection header received
0811 09:20:12.855763 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:12.855769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:12.855846 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:12.855935 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:12.857433 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.858111 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.859590 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.859997 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.860107 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:12.863617 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:12.863621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:12.863625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:12.863665 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:12.866273 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:12.866960 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:12.866964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:12.866966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:12.867084 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:12.867474 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:12.867478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:12.867480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:12.867763 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:12.867905 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:12.868805 (+ 900us) server_negotiation.cc:299] Negotiation successful
0811 09:20:12.869009 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:20:12.870682 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.852551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41019 (local address 127.0.0.1:45438)
0811 09:20:12.853345 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.853394 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:12.855487 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:12.856177 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:12.856186 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:12.856686 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:12.857269 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:12.857287 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.858268 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.858272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:12.859443 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:12.859456 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.860212 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.860220 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:12.860439 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:12.861153 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:12.861175 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:12.863409 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:12.866401 (+ 2992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:12.866410 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:12.866425 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:12.866838 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:12.867189 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:12.867194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:12.867197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:12.867362 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:12.867918 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:12.867928 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:12.868395 (+ 467us) client_negotiation.cc:769] Sending connection context
0811 09:20:12.868717 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 09:20:12.869290 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":312,"threads_started":1}
WARNING: no policy specified for kudu/127.17.16.124@KRBTEST.COM; defaulting to no policy
I20260811 09:20:12.872462 22758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.124@KRBTEST.COM" created.
Aug 11 09:20:12 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:20:12.887434 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.873348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35988)
0811 09:20:12.873499 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.873514 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:12.873597 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:12.874535 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:12.874539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:12.874996 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:12.875178 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:12.875190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.876543 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.876549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:12.877774 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:12.877790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.878574 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.878580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:12.878821 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:12.879378 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:12.879398 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:12.881575 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:12.884941 (+ 3366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:12.884946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:12.884949 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:12.885392 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:12.885743 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:12.885748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:12.885750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:12.885898 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:12.886507 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:12.886513 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:12.886646 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 09:20:12.886776 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 09:20:12.887062 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 09:20:12.887545 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.873565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.17.16.125:35225)
0811 09:20:12.874106 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:12.874112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:12.874140 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:20:12.874192 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:12.874199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:12.874393 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:12.874538 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:12.875359 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.876401 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.877908 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:12.878423 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:12.878515 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:12.881764 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:12.881777 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:12.881794 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:12.881830 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:12.884788 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:12.885512 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:12.885519 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:12.885522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:12.885642 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:12.885987 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:12.885991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:12.885993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:12.886364 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:12.886545 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:12.886927 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 09:20:12.887185 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":409,"thread_start_us":286,"threads_started":1}
I20260811 09:20:12.890380 22758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:20:12.891382 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.890975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36112)
0811 09:20:12.891116 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.891242 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 09:20:12.891620 22662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
I20260811 09:20:12.893072 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.892725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36122)
0811 09:20:12.892858 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.892970 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 09:20:12.893271 22758 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:45223: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab.
Entry for principal kudu/127.17.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab.
I20260811 09:20:12.911660 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.124:45223
--webserver_interface=127.17.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab
--principal=kudu/127.17.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:45173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:41019,127.17.16.125:35225,127.17.16.124:45223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:20:12.942586 22758 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } attempt: 1
I20260811 09:20:12.943709 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.943231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36130)
0811 09:20:12.943432 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.943543 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 09:20:12.944701 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:12.944389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36146)
0811 09:20:12.944504 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:12.944599 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 09:20:12.944936 22758 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:45223: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
I20260811 09:20:13.013177 22758 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } attempt: 2
I20260811 09:20:13.014303 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.013827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36158)
0811 09:20:13.014047 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.014142 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 09:20:13.015262 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.014923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36168)
0811 09:20:13.015077 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.015147 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 09:20:13.015463 22758 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:45223: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
I20260811 09:20:13.019677 22623 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } attempt: 5
Aug 11 09:20:13 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:20:13.033363 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.020350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.17.16.125:35225)
0811 09:20:13.020473 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.020477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.020499 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:20:13.020711 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.020714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.020788 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.020868 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.021862 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.022562 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.024200 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.024635 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.024727 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.028201 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.028207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.028210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.028241 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.030736 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.031431 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.031437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.031441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.031582 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.031960 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.031965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.031967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.032258 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.032407 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.032821 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.033073 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 09:20:13.033363 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.020214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:35992)
0811 09:20:13.020425 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.020447 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:13.020597 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:13.020958 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:13.020963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:13.021469 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:13.021732 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.021746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.022696 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.022702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.024070 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.024087 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.024776 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.024782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.025054 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.025682 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:13.025707 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:13.028019 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:13.030862 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.030868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.030872 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.031235 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.031696 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.031701 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.031705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.031851 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.032378 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:13.032383 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:13.032566 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 09:20:13.032698 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 09:20:13.033005 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 09:20:13.034350 22623 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:20:13.035331 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.034979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36176)
0811 09:20:13.035128 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.035199 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 09:20:13.036154 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.035844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36186)
0811 09:20:13.035971 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.036044 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 09:20:13.036337 22623 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:45223: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
I20260811 09:20:13.087555 22623 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } attempt: 1
I20260811 09:20:13.088713 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.088240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36196)
0811 09:20:13.088441 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.088533 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 09:20:13.089731 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.089429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36212)
0811 09:20:13.089557 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.089627 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 09:20:13.090327 22623 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:45223: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
I20260811 09:20:13.106662 22758 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } attempt: 3
I20260811 09:20:13.107757 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.107259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36216)
0811 09:20:13.107502 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.107599 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 09:20:13.108937 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.108471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36218)
0811 09:20:13.108713 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.108793 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260811 09:20:13.109153 22758 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:45223: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
I20260811 09:20:13.158569 22623 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } attempt: 2
I20260811 09:20:13.159812 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.159313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36220)
0811 09:20:13.159536 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.159649 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 09:20:13.160831 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.160442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36222)
0811 09:20:13.160619 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.160698 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 09:20:13.161036 22623 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:45223: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
I20260811 09:20:13.252377 22758 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } attempt: 4
I20260811 09:20:13.253597 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.253126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36228)
0811 09:20:13.253346 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.253440 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 09:20:13.254254 22767 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
I20260811 09:20:13.254521 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.254207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36244)
0811 09:20:13.254350 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.254418 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 09:20:13.254578 22767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:13.254722 22767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:13.254734 22758 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:45223: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
W20260811 09:20:13.254846 22767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:13.262256 22767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:13.262347 22767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:13.262440 22767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:13.262506 22767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:13.262598 22767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:13.267231 22623 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } attempt: 3
I20260811 09:20:13.268261 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.267818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36256)
0811 09:20:13.268017 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.268111 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:20:13.269258 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.268867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36270)
0811 09:20:13.269081 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.269157 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260811 09:20:13.269447 22623 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:45223: Network error: Client connection negotiation failed: client connection to 127.17.16.124:45223: connect: Connection refused (error 111)
I20260811 09:20:13.272799 22767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:45173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:41019,127.17.16.125:35225,127.17.16.124:45223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab
--principal=kudu/127.17.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.124:45223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:13.274114 22767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:13.275461 22767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:20:13.287086 22778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:13.287020 22782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:13.287063 22779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:13.287922 22767 server_base.cc:1034] running on GCE node
I20260811 09:20:13.288762 22767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:13.290138 22767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:13.291356 22767 hybrid_clock.cc:648] HybridClock initialized: now 1786440013291295 us; error 38 us; skew 500 ppm
I20260811 09:20:13.291682 22767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:13 dist-test-slave-ncp9 krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440013, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:13.294682 22767 init.cc:377] Logged in from keytab as kudu/127.17.16.124@KRBTEST.COM (short username kudu)
I20260811 09:20:13.296152 22767 webserver.cc:460] Webserver started at http://127.17.16.124:41691/ using document root <none> and password file <none>
I20260811 09:20:13.296864 22767 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:13.297004 22767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:13.297323 22767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:13.300323 22767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/data/instance:
uuid: "62c2aea7645041d9968bb19989ce020e"
format_stamp: "Formatted at 2026-08-11 09:20:13 on dist-test-slave-ncp9"
server_key: "a40dc0c5f23cb12bc194f5d307954fa4"
server_key_iv: "18a7c70ed52b8f1f08fa377bc4a72fed"
server_key_version: "encryptionkey@0"
I20260811 09:20:13.301149 22767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/wal/instance:
uuid: "62c2aea7645041d9968bb19989ce020e"
format_stamp: "Formatted at 2026-08-11 09:20:13 on dist-test-slave-ncp9"
server_key: "a40dc0c5f23cb12bc194f5d307954fa4"
server_key_iv: "18a7c70ed52b8f1f08fa377bc4a72fed"
server_key_version: "encryptionkey@0"
I20260811 09:20:13.306275 22767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:20:13.309725 22792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:13.311043 22767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:20:13.311205 22767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/wal
uuid: "62c2aea7645041d9968bb19989ce020e"
format_stamp: "Formatted at 2026-08-11 09:20:13 on dist-test-slave-ncp9"
server_key: "a40dc0c5f23cb12bc194f5d307954fa4"
server_key_iv: "18a7c70ed52b8f1f08fa377bc4a72fed"
server_key_version: "encryptionkey@0"
I20260811 09:20:13.311407 22767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:13.322654 22767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:13.326686 22767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:13.326936 22767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:13.344069 22767 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.124:45223
I20260811 09:20:13.344095 22876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.124:45223 every 8 connection(s)
I20260811 09:20:13.345440 22767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
I20260811 09:20:13.348984 22878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:13.352620 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 22767
I20260811 09:20:13.352807 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439649204149-17473-0/minicluster-data/master-2/wal/instance
I20260811 09:20:13.353374 17473 external_mini_cluster.cc:1428] Setting key 8e27eaefd8169b01ebbedff92dbf658e
I20260811 09:20:13.356479 22878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:20:13 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
Aug 11 09:20:13 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440013, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:20:13.374774 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.360244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.17.16.126:41019)
0811 09:20:13.360479 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.360485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.361211 (+ 726us) server_negotiation.cc:407] Connection header received
0811 09:20:13.361466 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.361471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.361536 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.361619 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.362718 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.363409 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.364768 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.365239 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.365350 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.368421 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.368428 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.368432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.368473 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.371628 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.372450 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.372456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.372459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.372627 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.373125 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.373130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.373133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.373473 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.373634 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.374171 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.374396 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:20:13.375759 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.355228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.17.16.124:45223)
0811 09:20:13.355986 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.355997 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.356038 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:20:13.357601 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.357625 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.358016 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.358344 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.360187 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.361829 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.363152 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.363818 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.363982 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.368168 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.368177 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.368189 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.368223 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.371672 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.372452 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.372460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.372463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.372657 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.373058 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.373063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.373066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.373774 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.374017 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.374397 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.374731 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":553,"thread_start_us":349,"threads_started":1}
I20260811 09:20:13.375800 22883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.360277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:41019 (local address 127.0.0.1:45454)
0811 09:20:13.361070 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.361123 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:13.361332 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:13.361704 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:13.361711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:13.362319 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:13.362572 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.362581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.363564 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.363568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.364625 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.364645 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.365454 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.365462 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.365600 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.366181 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:13.366201 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:13.368228 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:13.371764 (+ 3536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.371777 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.371788 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.372337 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.372758 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.372762 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.372765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.373008 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.373592 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:13.373598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:13.373802 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 09:20:13.374076 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 09:20:13.374584 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":346,"threads_started":1}
I20260811 09:20:13.377399 22878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:20:13 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440013, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
Aug 11 09:20:13 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:20:13.389842 22883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.378246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:35225 (local address 127.0.0.1:36008)
0811 09:20:13.378414 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.378431 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:13.378528 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:13.378874 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:13.378878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:13.379224 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:13.379353 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.379361 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.380306 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.380311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.381260 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.381273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.381950 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.381955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.382121 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.382647 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:13.382663 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:13.384658 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:13.387471 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.387476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.387481 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.387934 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.388299 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.388304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.388309 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.388448 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.388949 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:13.388953 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:13.389101 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 09:20:13.389229 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 09:20:13.389492 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:20:13.389906 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.378329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36008 (local address 127.17.16.125:35225)
0811 09:20:13.378526 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.378531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.378550 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:20:13.378667 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.378671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.378747 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.378846 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.379494 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.380176 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.381396 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.381810 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.381919 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.384830 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.384835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.384838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.384871 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.387344 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.388043 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.388048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.388051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.388194 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.388549 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.388552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.388554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.388816 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.388970 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.389337 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.389549 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:20:13.390888 22878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:20:13.392761 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.378920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45458 (local address 127.17.16.126:41019)
0811 09:20:13.379063 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.379067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.379249 (+ 182us) server_negotiation.cc:407] Connection header received
0811 09:20:13.379376 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.379381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.379442 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.379517 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.380632 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.381517 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.382814 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.383208 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.383313 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.387077 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.387082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.387085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.387115 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.389585 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.390626 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.390631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.390644 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.390801 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.391184 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.391188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.391190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.391513 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.391794 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.392245 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.392460 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 09:20:13 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440013, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
Aug 11 09:20:13 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:20:13.404037 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.391640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36290 (local address 127.17.16.124:45223)
0811 09:20:13.391770 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.391775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.391793 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:20:13.391937 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.391942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.391992 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.392079 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.392774 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.393627 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.395132 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.395515 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.395613 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.398782 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.398789 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.398794 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.398831 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.401415 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.402104 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.402109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.402113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.402271 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.402638 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.402643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.402646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.402992 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.403144 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.403450 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.403758 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":53}
I20260811 09:20:13.404037 22883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.391587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36290)
0811 09:20:13.391737 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.391755 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:13.391847 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:13.392109 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:13.392114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:13.392464 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:13.392623 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.392635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.393767 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.393772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.394963 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.394978 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.395691 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.395698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.395910 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.396466 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:13.396482 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:13.398594 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:13.401541 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.401546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.401550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.401986 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.402393 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.402398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.402402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.402527 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.403100 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:13.403103 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:13.403223 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 09:20:13.403358 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 09:20:13.403657 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 09:20:13.408706 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.394495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.17.16.125:35225)
0811 09:20:13.394677 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.394683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.394701 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:20:13.394751 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.394755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.394886 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.394982 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.396180 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.396984 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.398386 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.398899 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.399054 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.403400 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.403406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.403410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.403447 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.406342 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.407060 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.407064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.407066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.407192 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.407567 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.407570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.407572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.407820 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.408018 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.408280 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.408435 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:20:13.409729 22878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e: Bootstrap starting.
I20260811 09:20:13.413300 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:20:13.413295 22878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:13.414561 22878 log.cc:824] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:13.417405 22878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e: No bootstrap required, opened a new log
I20260811 09:20:13.416666 22579 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c43e3efb1ae94edda2cb932a425ae40c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45458
I20260811 09:20:13.424312 22878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } }
I20260811 09:20:13.424600 22878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c2aea7645041d9968bb19989ce020e, State: Initialized, Role: FOLLOWER
I20260811 09:20:13.425061 22878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } }
I20260811 09:20:13.425979 22886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } } }
I20260811 09:20:13.426362 22886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:13.426942 22878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:20:13.435659 22909 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:20:13.435789 22909 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:20:13.433944 22767 master_runner.cc:186] Error getting master registration for 127.17.16.126:41019: OK, instance_id { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" instance_seqno: 1786440012312854 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:20:13.444579 22767 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:20:13.446625 22623 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } attempt: 4
Aug 11 09:20:13 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
I20260811 09:20:13.464406 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.447363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36298)
0811 09:20:13.447682 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.447713 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:13.447837 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:13.448972 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:13.448993 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:13.449504 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:13.449716 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.449729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.451396 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.451402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.452852 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.452869 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.453950 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.453959 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.454230 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.454849 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:13.454884 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:13.457700 (+ 2816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:13.461567 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.461572 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.461575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.462067 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.462516 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.462521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.462524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.462703 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.463436 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:13.463440 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:13.463606 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 09:20:13.463722 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 09:20:13.464002 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260811 09:20:13.464417 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.447519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.17.16.124:45223)
0811 09:20:13.448565 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.448571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.448591 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:20:13.448664 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.448669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.448763 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.448870 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.449907 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.451226 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.453030 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.453769 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.453891 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.457917 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.457924 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.457932 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.457972 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.461418 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.462228 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.462234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.462238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.462388 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.462828 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.462833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.462836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.463295 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.463449 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.463862 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.464066 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933}
I20260811 09:20:13.469096 22623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c: Bootstrap starting.
I20260811 09:20:13.472162 22623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:13.473234 22623 log.cc:824] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:13.475771 22623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c: No bootstrap required, opened a new log
I20260811 09:20:13.480746 22623 raft_consensus.cc:348] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } }
I20260811 09:20:13.481208 22623 raft_consensus.cc:729] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43e3efb1ae94edda2cb932a425ae40c, State: Initialized, Role: FOLLOWER
I20260811 09:20:13.481781 22623 consensus_queue.cc:260] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } }
I20260811 09:20:13.482750 22918 sys_catalog.cc:455] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } } }
I20260811 09:20:13.483140 22918 sys_catalog.cc:458] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:13.483860 22623 sys_catalog.cc:564] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:20:13.484091 22579 raft_consensus.cc:482] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 09:20:13.484305 22579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:13.487756 22579 raft_consensus.cc:504] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } }
I20260811 09:20:13.491487 22579 leader_election.cc:288] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [CANDIDATE]: Term 1 election: Requested vote from peers 454e6ab6f26345c58aa4d0c0feb11b03 (127.17.16.125:35225), 62c2aea7645041d9968bb19989ce020e (127.17.16.124:45223)
I20260811 09:20:13.491348 22833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c43e3efb1ae94edda2cb932a425ae40c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "62c2aea7645041d9968bb19989ce020e"
I20260811 09:20:13.491868 22833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:13.495515 22833 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43e3efb1ae94edda2cb932a425ae40c in term 1.
I20260811 09:20:13.496271 22540 leader_election.cc:302] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62c2aea7645041d9968bb19989ce020e, c43e3efb1ae94edda2cb932a425ae40c; no voters:
I20260811 09:20:13.496726 22918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:20:13.505504 22918 raft_consensus.cc:686] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [term 1 LEADER]: Becoming Leader. State: Replica: c43e3efb1ae94edda2cb932a425ae40c, State: Running, Role: LEADER
I20260811 09:20:13.506157 22918 consensus_queue.cc:237] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } }
I20260811 09:20:13.519030 22946 sys_catalog.cc:455] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c43e3efb1ae94edda2cb932a425ae40c. Latest consensus state: current_term: 1 leader_uuid: "c43e3efb1ae94edda2cb932a425ae40c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } } }
I20260811 09:20:13.519479 22946 sys_catalog.cc:458] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:13.520576 22502 master_runner.cc:186] Error getting master registration for 127.17.16.125:35225: OK, instance_id { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" instance_seqno: 1786440012833125 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:20:13.531348 22502 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 09:20:13.535459 22942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:20:13.535642 22942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:20:13.535828 22952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:20:13.537988 22952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 09:20:13.552875 22544 tablet.cc:2369] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:20:13.553906 22758 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } attempt: 5
I20260811 09:20:13.562882 22833 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [term 1 FOLLOWER]: Refusing update from remote peer c43e3efb1ae94edda2cb932a425ae40c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:20:13.564134 22946 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 11 09:20:13 dist-test-slave-ncp9 krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440012, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
W20260811 09:20:13.578809 22800 tablet.cc:2369] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:20:13.585688 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.554548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:45223 (local address 127.0.0.1:36300)
0811 09:20:13.555024 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:13.555043 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:13.555156 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:13.556081 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:13.556086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:13.556610 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:13.556834 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.556847 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.558428 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.558434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.559869 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:13.559886 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.562855 (+ 2969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.562861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:13.563300 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.569131 (+ 5831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:13.569160 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:13.573210 (+ 4050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:13.582439 (+ 9229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.582445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.582449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.583009 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.583599 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:13.583605 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:13.583609 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:13.583826 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:13.584583 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:13.584588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:13.584758 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 09:20:13.584918 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 09:20:13.585240 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342}
I20260811 09:20:13.587399 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.555390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.17.16.124:45223)
0811 09:20:13.555567 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.555574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.555592 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:20:13.555674 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.555679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.555774 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.555872 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.557065 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.558250 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.560055 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.560768 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.560917 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.573545 (+ 12628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.573556 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.573561 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.573612 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.580754 (+ 7142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.583264 (+ 2510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.583270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.583274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.583448 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.584048 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.584053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.584057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.584424 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.586404 (+ 1980us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.586550 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.587040 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:20:13.593814 22758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03: Bootstrap starting.
I20260811 09:20:13.603808 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.533848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36030 (local address 127.17.16.125:35225)
0811 09:20:13.534030 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.534035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.572242 (+ 38207us) server_negotiation.cc:407] Connection header received
0811 09:20:13.572736 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.572743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.572828 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.572926 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.576123 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.577247 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.580619 (+ 3372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.581269 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.581399 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.594850 (+ 13451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.594859 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.594864 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.594908 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.598596 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.601139 (+ 2543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.601145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.601149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.601332 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.601854 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.601860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.601864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.602267 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.602515 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.603109 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.603371 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:20:13.604287 22758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:13.618834 22758 log.cc:824] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:13.620055 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.572069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36316 (local address 127.17.16.124:45223)
0811 09:20:13.572748 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.572753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.579816 (+ 7063us) server_negotiation.cc:407] Connection header received
0811 09:20:13.579877 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.579882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.579970 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.580088 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.592469 (+ 12381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.593845 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.596674 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.597401 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.597514 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.606049 (+ 8535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.606057 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.606061 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.606106 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.612457 (+ 6351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.613503 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.613509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.613513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.613677 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.614113 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.614118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.614122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.614646 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.616105 (+ 1459us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.616242 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.616542 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":2341,"threads_started":1}
I20260811 09:20:13.623553 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.533851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45474 (local address 127.17.16.126:41019)
0811 09:20:13.534017 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.534024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.542370 (+ 8346us) server_negotiation.cc:407] Connection header received
0811 09:20:13.571546 (+ 29176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.571552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.571672 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.571777 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.580675 (+ 8898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.581886 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.585111 (+ 3225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:13.585788 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:13.585898 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:13.605115 (+ 19217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:13.605123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:13.605129 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:13.605179 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:13.609311 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.619179 (+ 9868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.619188 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.619192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.619410 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:13.620479 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:13.620486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:13.620490 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:13.621008 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:13.621301 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:13.622834 (+ 1533us) server_negotiation.cc:299] Negotiation successful
0811 09:20:13.623140 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:20:13.628703 22886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c43e3efb1ae94edda2cb932a425ae40c. Latest consensus state: current_term: 1 leader_uuid: "c43e3efb1ae94edda2cb932a425ae40c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } } }
I20260811 09:20:13.629066 22886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:13.631580 22957 mvcc.cc:205] Tried to move back new op lower bound from 7317258295536205824 to 7317258295327346688. Current Snapshot: MvccSnapshot[applied={T|T < 7317258295536205824}]
I20260811 09:20:13.635476 22758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03: No bootstrap required, opened a new log
I20260811 09:20:13.643110 22886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c43e3efb1ae94edda2cb932a425ae40c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } } }
I20260811 09:20:13.643388 22886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:13.647747 22918 sys_catalog.cc:455] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c43e3efb1ae94edda2cb932a425ae40c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } } }
I20260811 09:20:13.648877 22918 sys_catalog.cc:458] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:13.648929 22758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } }
I20260811 09:20:13.649364 22758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 454e6ab6f26345c58aa4d0c0feb11b03, State: Initialized, Role: FOLLOWER
I20260811 09:20:13.649911 22758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } }
I20260811 09:20:13.655143 22952 catalog_manager.cc:1347] Generated new cluster ID: 4dafed4865484d87beba898b1b4a3308
I20260811 09:20:13.655238 22952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:20:13.680655 22758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:20:13.681172 22705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:13.684998 22707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c43e3efb1ae94edda2cb932a425ae40c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "454e6ab6f26345c58aa4d0c0feb11b03"
I20260811 09:20:13.685738 22705 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [term 1 FOLLOWER]: Refusing update from remote peer c43e3efb1ae94edda2cb932a425ae40c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:20:13.689275 22707 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43e3efb1ae94edda2cb932a425ae40c in term 1.
I20260811 09:20:13.690886 22918 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [LEADER]: Connected to new peer: Peer: permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:20:13.680160 22972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } } }
I20260811 09:20:13.695514 22972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:13.721060 22972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c43e3efb1ae94edda2cb932a425ae40c. Latest consensus state: current_term: 1 leader_uuid: "c43e3efb1ae94edda2cb932a425ae40c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } } }
I20260811 09:20:13.721565 22972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:13.721405 22974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c43e3efb1ae94edda2cb932a425ae40c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } } }
I20260811 09:20:13.723278 22974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:13.743266 22975 mvcc.cc:205] Tried to move back new op lower bound from 7317258295536205824 to 7317258295327346688. Current Snapshot: MvccSnapshot[applied={T|T < 7317258295536205824}]
I20260811 09:20:13.750105 22918 sys_catalog.cc:455] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c43e3efb1ae94edda2cb932a425ae40c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43e3efb1ae94edda2cb932a425ae40c" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 41019 } } peers { permanent_uuid: "454e6ab6f26345c58aa4d0c0feb11b03" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 35225 } } peers { permanent_uuid: "62c2aea7645041d9968bb19989ce020e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 45223 } } }
I20260811 09:20:13.750413 22918 sys_catalog.cc:458] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:13.760720 22952 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:20:13.763087 22952 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:20:13.786795 22952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c43e3efb1ae94edda2cb932a425ae40c: Generated new TSK 0
I20260811 09:20:13.788252 22952 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:20:13.820711 22998 catalog_manager.cc:1259] Loaded cluster ID: 4dafed4865484d87beba898b1b4a3308
I20260811 09:20:13.820849 22998 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03: loading cluster ID for follower catalog manager: success
I20260811 09:20:13.825261 22998 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03: acquiring CA information for follower catalog manager: success
I20260811 09:20:13.828096 22998 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 454e6ab6f26345c58aa4d0c0feb11b03: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 09:20:13.945663 22627 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:13.931340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.17.16.126:41019)
0811 09:20:13.931595 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.931602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.942850 (+ 11248us) server_negotiation.cc:407] Connection header received
0811 09:20:13.943918 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.943926 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.944042 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.944125 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.945395 (+ 1270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45478: BlockingRecv error: recv got EOF from 127.0.0.1:45478 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260811 09:20:13.949719 22764 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:13.938648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.17.16.125:35225)
0811 09:20:13.938863 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.938869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.947370 (+ 8501us) server_negotiation.cc:407] Connection header received
0811 09:20:13.948160 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.948166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.948279 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.948359 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.949460 (+ 1101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36046: BlockingRecv error: recv got EOF from 127.0.0.1:36046 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 09:20:13.953145 22962 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:13.943437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36332 (local address 127.17.16.124:45223)
0811 09:20:13.943646 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.943652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:13.951101 (+ 7449us) server_negotiation.cc:407] Connection header received
0811 09:20:13.951762 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:13.951769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:13.951858 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:13.951931 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:13.952869 (+ 938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36332: BlockingRecv error: recv got EOF from 127.0.0.1:36332 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:20:14.010913 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:13.995966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36056 (local address 127.17.16.125:35225)
0811 09:20:13.996174 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:13.996181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.003053 (+ 6872us) server_negotiation.cc:407] Connection header received
0811 09:20:14.003626 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.003632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.003851 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.004006 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.004708 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.005836 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.007827 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.008537 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.008683 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.009314 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.010030 (+ 716us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.010152 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.010408 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.010530 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 09:20:14.047086 22962 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:14.044124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36340 (local address 127.17.16.124:45223)
0811 09:20:14.044307 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.044314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.045590 (+ 1276us) server_negotiation.cc:407] Connection header received
0811 09:20:14.045814 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.045820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.045915 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.046015 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:14.046816 (+ 801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36340: BlockingRecv error: recv got EOF from 127.0.0.1:36340 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:20:14.067888 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.062629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36060 (local address 127.17.16.125:35225)
0811 09:20:14.062855 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.062863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.063330 (+ 467us) server_negotiation.cc:407] Connection header received
0811 09:20:14.063544 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.063548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.063617 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.063692 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.064108 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.064861 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.066216 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.066654 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.066769 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.067158 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.067290 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.067363 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.067503 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.067584 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 09:20:14.076157 22962 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:14.074517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.17.16.124:45223)
0811 09:20:14.074705 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.074711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.074816 (+ 105us) server_negotiation.cc:407] Connection header received
0811 09:20:14.075072 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.075078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.075154 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.075237 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:14.075988 (+ 751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36342: BlockingRecv error: recv got EOF from 127.0.0.1:36342 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:20:14.087232 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.081630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36074 (local address 127.17.16.125:35225)
0811 09:20:14.081814 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.081820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.082314 (+ 494us) server_negotiation.cc:407] Connection header received
0811 09:20:14.082518 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.082524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.082594 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.082677 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.083113 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.083901 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.085366 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.085827 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.085941 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.086345 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.086496 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.086603 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.086806 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.086896 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 09:20:14.098706 22962 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:14.093148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36346 (local address 127.17.16.124:45223)
0811 09:20:14.093323 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.093330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.097346 (+ 4016us) server_negotiation.cc:407] Connection header received
0811 09:20:14.097517 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.097522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.097611 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.097703 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:14.098453 (+ 750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36346: BlockingRecv error: recv got EOF from 127.0.0.1:36346 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 09:20:14.121667 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.116308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36078 (local address 127.17.16.125:35225)
0811 09:20:14.116639 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.116645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.117164 (+ 519us) server_negotiation.cc:407] Connection header received
0811 09:20:14.117352 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.117356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.117423 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.117497 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.117902 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.118669 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.119977 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.120453 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.120561 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.120983 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.121091 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.121170 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.121311 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.121394 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
W20260811 09:20:14.129783 22962 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:14.127508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.17.16.124:45223)
0811 09:20:14.127780 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.127787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.128287 (+ 500us) server_negotiation.cc:407] Connection header received
0811 09:20:14.128514 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.128519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.128592 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.128671 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:14.129535 (+ 864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36362: BlockingRecv error: recv got EOF from 127.0.0.1:36362 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
I20260811 09:20:14.147090 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.141517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36082 (local address 127.17.16.125:35225)
0811 09:20:14.141720 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.141727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.142219 (+ 492us) server_negotiation.cc:407] Connection header received
0811 09:20:14.142412 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.142416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.142487 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.142563 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.142992 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.143825 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.145242 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.145718 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.145798 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.146288 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.146451 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.146532 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.146699 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.146775 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 09:20:14.154765 22962 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:14.152669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36378 (local address 127.17.16.124:45223)
0811 09:20:14.152881 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.152908 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.153416 (+ 508us) server_negotiation.cc:407] Connection header received
0811 09:20:14.153572 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.153578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.153658 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.153740 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:14.154522 (+ 782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36378: BlockingRecv error: recv got EOF from 127.0.0.1:36378 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:20:14.188143 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.182485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36094 (local address 127.17.16.125:35225)
0811 09:20:14.182691 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.182698 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.183228 (+ 530us) server_negotiation.cc:407] Connection header received
0811 09:20:14.183435 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.183440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.183516 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.183592 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.184103 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.185030 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.186399 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.186860 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.186978 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.187402 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.187521 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.187595 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.187765 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.187854 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 09:20:14.195940 22962 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:14.194222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.17.16.124:45223)
0811 09:20:14.194412 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.194419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.194487 (+ 68us) server_negotiation.cc:407] Connection header received
0811 09:20:14.194757 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.194763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.194848 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.194933 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:14.195720 (+ 787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36394: BlockingRecv error: recv got EOF from 127.0.0.1:36394 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:20:14.236784 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.231025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36098 (local address 127.17.16.125:35225)
0811 09:20:14.231258 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.231265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.231624 (+ 359us) server_negotiation.cc:407] Connection header received
0811 09:20:14.231805 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.231811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.231908 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.232066 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.232401 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.233605 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.234923 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.235577 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.235715 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.236058 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.236200 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.236304 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.236377 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.236462 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 09:20:14.244043 22962 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:14.242220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36404 (local address 127.17.16.124:45223)
0811 09:20:14.242433 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.242439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.242717 (+ 278us) server_negotiation.cc:407] Connection header received
0811 09:20:14.242860 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.242867 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.242976 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.243095 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:14.243784 (+ 689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36404: BlockingRecv error: recv got EOF from 127.0.0.1:36404 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:20:14.316738 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.311215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36108 (local address 127.17.16.125:35225)
0811 09:20:14.311441 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.311448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.311761 (+ 313us) server_negotiation.cc:407] Connection header received
0811 09:20:14.311927 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.311933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.312036 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.312171 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.312528 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.313652 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.314952 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.315571 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.315691 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.316069 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.316184 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.316251 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.316405 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.316466 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 09:20:14.325464 22962 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:14.322952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.17.16.124:45223)
0811 09:20:14.323137 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.323143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.324052 (+ 909us) server_negotiation.cc:407] Connection header received
0811 09:20:14.324265 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.324269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.324345 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.324427 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:14.325258 (+ 831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36412: BlockingRecv error: recv got EOF from 127.0.0.1:36412 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:20:14.437856 22909 catalog_manager.cc:1259] Loaded cluster ID: 4dafed4865484d87beba898b1b4a3308
I20260811 09:20:14.437958 22909 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e: loading cluster ID for follower catalog manager: success
I20260811 09:20:14.440425 22909 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e: acquiring CA information for follower catalog manager: success
I20260811 09:20:14.442209 22909 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 62c2aea7645041d9968bb19989ce020e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:20:14.462368 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.457663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.17.16.125:35225)
0811 09:20:14.457899 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.457907 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.458124 (+ 217us) server_negotiation.cc:407] Connection header received
0811 09:20:14.458301 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.458307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.458382 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.458486 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.458880 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.459611 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.460863 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.461347 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.461444 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.461766 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.461877 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.461940 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.462019 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.462080 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 09:20:14.473297 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.467859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.17.16.124:45223)
0811 09:20:14.468074 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.468080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.468316 (+ 236us) server_negotiation.cc:407] Connection header received
0811 09:20:14.468492 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.468497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.468649 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.468750 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.469171 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.469912 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.471200 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.471589 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.471692 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.472179 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.472678 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.472813 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.472943 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.473036 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:20:14.487768 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.479511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36122 (local address 127.17.16.125:35225)
0811 09:20:14.479716 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.479723 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.481716 (+ 1993us) server_negotiation.cc:407] Connection header received
0811 09:20:14.481899 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.481905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.481991 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.482086 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.482858 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.483827 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.485717 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.486378 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.486512 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.487018 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.487149 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.487234 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.487377 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.487459 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:20:14.488266 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.479232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45494 (local address 127.17.16.126:41019)
0811 09:20:14.481289 (+ 2057us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.481296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.481318 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:20:14.481446 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.481452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.481785 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.482704 (+ 919us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.482879 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.483801 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.485795 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.486398 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.486745 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.487148 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.487599 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.487759 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.487871 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.487953 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1921,"thread_start_us":1817,"threads_started":1}
I20260811 09:20:14.488618 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:14.480013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36430 (local address 127.17.16.124:45223)
0811 09:20:14.480206 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:14.480213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:14.482492 (+ 2279us) server_negotiation.cc:407] Connection header received
0811 09:20:14.482718 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:14.482721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:14.482783 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:14.482861 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:14.483401 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.484419 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.486287 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:14.486833 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:14.486955 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:14.487786 (+ 831us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:14.487932 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:14.488019 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:14.488261 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 09:20:14.488347 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:20:14.493371 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22502
I20260811 09:20:14.494681 22613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:20:14.607640 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 22502
I20260811 09:20:14.638667 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22636
I20260811 09:20:14.639809 22751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:20:14.746624 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 22636
I20260811 09:20:14.768551 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 22767
I20260811 09:20:14.769752 22868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:20:14.876433 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 22767
2026-08-11T09:20:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4578 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:20:14 dist-test-slave-ncp9 krb5kdc[23206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:20:14 dist-test-slave-ncp9 krb5kdc[23206](info): set up 2 sockets
Aug 11 09:20:14 dist-test-slave-ncp9 krb5kdc[23206](info): commencing operation
krb5kdc: starting...
W20260811 09:20:16.134406 17473 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/test-user.keytab.
Aug 11 09:20:16 dist-test-slave-ncp9 krb5kdc[23206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:20:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:20:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.16.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
Entry for principal kudu/127.17.16.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab.
I20260811 09:20:16.459571 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:37419
--webserver_interface=127.17.16.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:34521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:37419,127.17.16.125:39551,127.17.16.124:40453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:20:16.802742 23229 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:20:16.803082 23229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:16.803162 23229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:16.803251 23229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:16.810763 23229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:16.810868 23229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:16.810940 23229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:16.810998 23229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:16.811069 23229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:16.821173 23229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:34521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:37419,127.17.16.125:39551,127.17.16.124:40453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.126.keytab
--principal=kudu/127.17.16.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.126:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.16.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:16.822510 23229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:16.823902 23229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:20:16.836167 23234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:16.836078 23249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:16.836125 23239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:16.837194 23229 server_base.cc:1034] running on GCE node
I20260811 09:20:16.837963 23229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:16.839366 23229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:16.840601 23229 hybrid_clock.cc:648] HybridClock initialized: now 1786440016840532 us; error 35 us; skew 500 ppm
I20260811 09:20:16.841005 23229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:16 dist-test-slave-ncp9 krb5kdc[23206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440016, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:16.844194 23229 init.cc:377] Logged in from keytab as kudu/127.17.16.126@KRBTEST.COM (short username kudu)
I20260811 09:20:16.845690 23229 webserver.cc:460] Webserver started at http://127.17.16.126:41861/ using document root <none> and password file <none>
I20260811 09:20:16.846365 23229 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:16.846462 23229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:16.846782 23229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:16.849705 23229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/data/instance:
uuid: "5d85c85be6bb46ccbddd0f58370842cb"
format_stamp: "Formatted at 2026-08-11 09:20:16 on dist-test-slave-ncp9"
server_key: "b6f900abe0a085727fb1e9a9a05745f0"
server_key_iv: "5db962adf8748cb1c3cda187ed60b945"
server_key_version: "encryptionkey@0"
I20260811 09:20:16.850497 23229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/wal/instance:
uuid: "5d85c85be6bb46ccbddd0f58370842cb"
format_stamp: "Formatted at 2026-08-11 09:20:16 on dist-test-slave-ncp9"
server_key: "b6f900abe0a085727fb1e9a9a05745f0"
server_key_iv: "5db962adf8748cb1c3cda187ed60b945"
server_key_version: "encryptionkey@0"
I20260811 09:20:16.855396 23229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:20:16.858714 23261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:16.860045 23229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:20:16.860234 23229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/wal
uuid: "5d85c85be6bb46ccbddd0f58370842cb"
format_stamp: "Formatted at 2026-08-11 09:20:16 on dist-test-slave-ncp9"
server_key: "b6f900abe0a085727fb1e9a9a05745f0"
server_key_iv: "5db962adf8748cb1c3cda187ed60b945"
server_key_version: "encryptionkey@0"
I20260811 09:20:16.860459 23229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:16.875918 23229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:16.879932 23229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:16.880214 23229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:16.899179 23229 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.126:37419
I20260811 09:20:16.899199 23345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.126:37419 every 8 connection(s)
I20260811 09:20:16.900684 23229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/data/info.pb
I20260811 09:20:16.901114 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 23229
I20260811 09:20:16.901279 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-0/wal/instance
I20260811 09:20:16.901778 17473 external_mini_cluster.cc:1428] Setting key 9cd32a81ca8aaf58559bc3838a7d6fda
I20260811 09:20:16.904803 23347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:16.912168 23347 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:20:16 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440016, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.16.125@KRBTEST.COM; defaulting to no policy
I20260811 09:20:16.939101 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:16.915827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37419 (local address 127.0.0.1:34284)
0811 09:20:16.919050 (+ 3223us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:16.919111 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:16.921621 (+ 2510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:16.922531 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:16.922539 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:16.923361 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:16.923837 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:16.923862 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:16.925513 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:16.925519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:16.927209 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:16.927228 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:16.928173 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:16.928181 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:16.928369 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:16.929109 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:16.929140 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:16.931912 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:16.934947 (+ 3035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:16.934960 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:16.934979 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:16.935599 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:16.935997 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:16.936001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:16.936005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:16.936221 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:16.936983 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:16.936992 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:16.937251 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 09:20:16.937557 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 09:20:16.938157 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2959,"mutex_wait_us":102,"thread_start_us":2743,"threads_started":1}
I20260811 09:20:16.939079 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:16.915826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34284 (local address 127.17.16.126:37419)
0811 09:20:16.918696 (+ 2870us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:16.918716 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:16.919250 (+ 534us) server_negotiation.cc:407] Connection header received
0811 09:20:16.921993 (+ 2743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:16.922000 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:16.922265 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:16.922415 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:16.924079 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:16.925332 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:16.927365 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:16.927908 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:16.928034 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:16.932144 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:16.932153 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:16.932164 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:16.932191 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:16.934771 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:16.935728 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:16.935735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:16.935737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:16.935884 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:16.936321 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:16.936324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:16.936326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:16.936816 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:16.937010 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:16.937670 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 09:20:16.937971 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":2610,"spinlock_wait_cycles":14080,"thread_start_us":2381,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.125@KRBTEST.COM" created.
I20260811 09:20:16.942095 23347 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:20:16.943157 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:16.942698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50518)
0811 09:20:16.942861 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:16.943003 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:20:16.943409 23263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
I20260811 09:20:16.944929 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:16.944518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50530)
0811 09:20:16.944669 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:16.944749 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 09:20:16.945194 23347 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:39551: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab.
Entry for principal kudu/127.17.16.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab.
I20260811 09:20:16.978363 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.125:39551
--webserver_interface=127.17.16.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab
--principal=kudu/127.17.16.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:34521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:37419,127.17.16.125:39551,127.17.16.124:40453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:20:16.994520 23347 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } attempt: 1
I20260811 09:20:16.995724 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:16.995265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50532)
0811 09:20:16.995472 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:16.995590 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 09:20:16.996701 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:16.996344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50540)
0811 09:20:16.996492 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:16.996568 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 09:20:16.996912 23347 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:39551: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
I20260811 09:20:17.065163 23347 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } attempt: 2
I20260811 09:20:17.066339 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.065851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50546)
0811 09:20:17.066106 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.066197 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 09:20:17.067370 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.067008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50548)
0811 09:20:17.067160 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.067230 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 09:20:17.067545 23347 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:39551: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
I20260811 09:20:17.158790 23347 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } attempt: 3
I20260811 09:20:17.159898 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.159450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50562)
0811 09:20:17.159668 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.159758 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 09:20:17.161005 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.160612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50578)
0811 09:20:17.160759 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.160825 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 09:20:17.161180 23347 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:39551: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
I20260811 09:20:17.304440 23347 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } attempt: 4
I20260811 09:20:17.305768 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.305211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50590)
0811 09:20:17.305537 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.305629 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 09:20:17.306775 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.306413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50598)
0811 09:20:17.306568 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.306649 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:20:17.306985 23347 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.125:39551: Network error: Client connection negotiation failed: client connection to 127.17.16.125:39551: connect: Connection refused (error 111)
W20260811 09:20:17.315191 23356 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:20:17.315505 23356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:17.315598 23356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:17.315709 23356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:17.322875 23356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:17.322963 23356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:17.323048 23356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:17.323118 23356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:17.323203 23356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:17.332906 23356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:34521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:37419,127.17.16.125:39551,127.17.16.124:40453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.125.keytab
--principal=kudu/127.17.16.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.125:39551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.16.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:17.334229 23356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:17.335601 23356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:20:17.346951 23365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:17.347002 23369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:17.347013 23366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:17.347752 23356 server_base.cc:1034] running on GCE node
I20260811 09:20:17.348505 23356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:17.349828 23356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:17.351042 23356 hybrid_clock.cc:648] HybridClock initialized: now 1786440017350970 us; error 42 us; skew 500 ppm
I20260811 09:20:17.351377 23356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440017, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:17.359514 23356 init.cc:377] Logged in from keytab as kudu/127.17.16.125@KRBTEST.COM (short username kudu)
I20260811 09:20:17.360980 23356 webserver.cc:460] Webserver started at http://127.17.16.125:39727/ using document root <none> and password file <none>
I20260811 09:20:17.361654 23356 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:17.361757 23356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:17.362067 23356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:17.365161 23356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/data/instance:
uuid: "ee0a4cc6a61c44dab72277817a6dc437"
format_stamp: "Formatted at 2026-08-11 09:20:17 on dist-test-slave-ncp9"
server_key: "fce9449000ba82e31ffd613ccb396f27"
server_key_iv: "b213985f3271f454f9dd96ca337044a8"
server_key_version: "encryptionkey@0"
I20260811 09:20:17.365953 23356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/wal/instance:
uuid: "ee0a4cc6a61c44dab72277817a6dc437"
format_stamp: "Formatted at 2026-08-11 09:20:17 on dist-test-slave-ncp9"
server_key: "fce9449000ba82e31ffd613ccb396f27"
server_key_iv: "b213985f3271f454f9dd96ca337044a8"
server_key_version: "encryptionkey@0"
I20260811 09:20:17.370924 23356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:20:17.374208 23384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:17.375433 23356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:20:17.375589 23356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/wal
uuid: "ee0a4cc6a61c44dab72277817a6dc437"
format_stamp: "Formatted at 2026-08-11 09:20:17 on dist-test-slave-ncp9"
server_key: "fce9449000ba82e31ffd613ccb396f27"
server_key_iv: "b213985f3271f454f9dd96ca337044a8"
server_key_version: "encryptionkey@0"
I20260811 09:20:17.375782 23356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:17.387720 23356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:17.391269 23356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:17.391499 23356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:17.408202 23356 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.125:39551
I20260811 09:20:17.408224 23473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.125:39551 every 8 connection(s)
I20260811 09:20:17.409535 23356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/data/info.pb
I20260811 09:20:17.411890 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 23356
I20260811 09:20:17.412088 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-1/wal/instance
I20260811 09:20:17.412645 17473 external_mini_cluster.cc:1428] Setting key d6c36eba2a90a8c935d74b16e113450d
I20260811 09:20:17.413630 23476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:17.421952 23476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440017, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:20:17.442179 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.424993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.17.16.126:37419)
0811 09:20:17.425189 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:17.425194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:17.426440 (+ 1246us) server_negotiation.cc:407] Connection header received
0811 09:20:17.428876 (+ 2436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:17.428883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:17.428988 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:17.429079 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:17.430587 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.431411 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.432870 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.433316 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.433435 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.436676 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:17.436681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:17.436685 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:17.436715 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:17.439279 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.439960 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.439966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.439968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.440087 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.440488 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.440494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.440496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.440780 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:17.440935 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:17.441629 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 09:20:17.441866 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:20:17.443352 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.425472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37419 (local address 127.0.0.1:34300)
0811 09:20:17.426268 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.426323 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:17.428518 (+ 2195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:17.429330 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:17.429338 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:17.429837 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:17.430400 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.430419 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.431560 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.431564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.432726 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.432739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.433538 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.433546 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.433765 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.434376 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:17.434403 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:17.436458 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:17.439410 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.439419 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.439434 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.439847 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.440211 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.440216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.440219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.440383 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.440950 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:17.440958 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:17.441279 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 09:20:17.441527 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 09:20:17.442004 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":306,"threads_started":1}
WARNING: no policy specified for kudu/127.17.16.124@KRBTEST.COM; defaulting to no policy
I20260811 09:20:17.445299 23476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.16.124@KRBTEST.COM" created.
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440017, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:20:17.460467 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.446017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50610)
0811 09:20:17.446283 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.446305 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:17.446405 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:17.447540 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:17.447545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:17.447969 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:17.448151 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.448163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.449770 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.449775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.451070 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.451084 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.451861 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.451867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.452025 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.452554 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:17.452571 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:17.454834 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:17.457821 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.457827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.457831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.458324 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.458664 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.458669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.458672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.458871 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.459448 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:17.459453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:17.459609 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 09:20:17.459768 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 09:20:17.460052 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 09:20:17.460482 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.446315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.17.16.125:39551)
0811 09:20:17.447094 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:17.447101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:17.447133 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:20:17.447183 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:17.447190 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:17.447405 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:17.447545 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:17.448316 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.449600 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.451199 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.451726 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.451835 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.455039 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:17.455050 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:17.455064 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:17.455096 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:17.457691 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.458446 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.458453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.458455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.458562 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.458975 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.458978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.458980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.459312 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:17.459464 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:17.459879 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 09:20:17.460167 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":603,"thread_start_us":433,"threads_started":1}
I20260811 09:20:17.463055 23476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:20:17.464154 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.463687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:53980)
0811 09:20:17.463833 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.463965 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:20:17.464427 23386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
I20260811 09:20:17.466105 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.465741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:53982)
0811 09:20:17.465904 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.465992 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 09:20:17.466329 23476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40453: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.16.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab.
Entry for principal kudu/127.17.16.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab.
I20260811 09:20:17.488157 17473 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
/tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.124:40453
--webserver_interface=127.17.16.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab
--principal=kudu/127.17.16.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.16.84:34521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.16.126:37419,127.17.16.125:39551,127.17.16.124:40453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:20:17.515679 23476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } attempt: 1
I20260811 09:20:17.516990 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.516406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:53984)
0811 09:20:17.516636 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.516772 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 09:20:17.517962 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.517641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:53988)
0811 09:20:17.517787 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.517860 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:20:17.518241 23476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40453: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
I20260811 09:20:17.586440 23476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } attempt: 2
I20260811 09:20:17.587512 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.587076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54002)
0811 09:20:17.587272 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.587359 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 09:20:17.588562 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.588241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54008)
0811 09:20:17.588368 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.588437 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 09:20:17.588764 23476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40453: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
I20260811 09:20:17.606213 23347 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } attempt: 5
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440016, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:20:17.619283 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.607058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50622 (local address 127.17.16.125:39551)
0811 09:20:17.607235 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:17.607241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:17.607272 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:20:17.607337 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:17.607342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:17.607418 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:17.607509 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:17.608389 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.609227 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.610373 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.610821 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.610931 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.614194 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:17.614199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:17.614202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:17.614236 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:17.616808 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.617477 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.617482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.617484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.617605 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.617961 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.617964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.617966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.618248 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:17.618380 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:17.618762 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 09:20:17.618959 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:20:17.619280 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.606867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50622)
0811 09:20:17.607062 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.607080 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:17.607213 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:17.607679 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:17.607683 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:17.608053 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:17.608239 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.608249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.609366 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.609371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.610221 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.610234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.610967 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.610972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.611180 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.611780 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:17.611797 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:17.614012 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:17.616958 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.616964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.616967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.617361 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.617721 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.617725 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.617727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.617855 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.618364 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:17.618369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:17.618512 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 09:20:17.618635 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 09:20:17.618918 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 09:20:17.620463 23347 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:20:17.621555 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.621123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54018)
0811 09:20:17.621297 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.621389 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 09:20:17.622529 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.622161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54028)
0811 09:20:17.622303 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.622407 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:20:17.622725 23347 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40453: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
I20260811 09:20:17.673977 23347 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } attempt: 1
I20260811 09:20:17.675340 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.674763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54036)
0811 09:20:17.674992 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.675115 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 09:20:17.676357 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.675988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54046)
0811 09:20:17.676132 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.676227 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 09:20:17.676556 23347 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40453: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
I20260811 09:20:17.679944 23476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } attempt: 3
I20260811 09:20:17.681005 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.680516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54062)
0811 09:20:17.680745 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.680839 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 09:20:17.682060 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.681634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54078)
0811 09:20:17.681863 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.681941 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260811 09:20:17.682286 23476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40453: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
I20260811 09:20:17.744832 23347 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } attempt: 2
I20260811 09:20:17.746173 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.745597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54084)
0811 09:20:17.745844 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.745956 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 09:20:17.747305 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.746888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54100)
0811 09:20:17.747047 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.747160 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:20:17.747579 23347 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40453: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
I20260811 09:20:17.825475 23476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } attempt: 4
I20260811 09:20:17.826578 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.826090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54114)
0811 09:20:17.826312 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.826397 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 09:20:17.827600 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.827286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54118)
0811 09:20:17.827403 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.827474 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 09:20:17.827802 23476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40453: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
W20260811 09:20:17.831655 23489 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:20:17.832047 23489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:20:17.832165 23489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:20:17.832280 23489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:20:17.840049 23489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:20:17.840153 23489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:20:17.840211 23489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:20:17.840260 23489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:20:17.840324 23489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:20:17.850620 23489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.16.84:34521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.16.126:37419,127.17.16.125:39551,127.17.16.124:40453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/krb5kdc/kudu_127.17.16.124.keytab
--principal=kudu/127.17.16.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.16.124:40453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.16.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:20:17.851845 23489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:20:17.853252 23489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 09:20:17.853801 23347 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } attempt: 3
I20260811 09:20:17.854933 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.854393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54120)
0811 09:20:17.854637 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.854742 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 09:20:17.855947 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.855575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54124)
0811 09:20:17.855740 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.855828 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 09:20:17.856158 23347 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.16.124:40453: Network error: Client connection negotiation failed: client connection to 127.17.16.124:40453: connect: Connection refused (error 111)
W20260811 09:20:17.865415 23498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:17.865453 23501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:20:17.865475 23497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:20:17.866250 23489 server_base.cc:1034] running on GCE node
I20260811 09:20:17.866955 23489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:20:17.868263 23489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:20:17.869470 23489 hybrid_clock.cc:648] HybridClock initialized: now 1786440017869400 us; error 42 us; skew 500 ppm
I20260811 09:20:17.869777 23489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440017, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:20:17.872807 23489 init.cc:377] Logged in from keytab as kudu/127.17.16.124@KRBTEST.COM (short username kudu)
I20260811 09:20:17.874307 23489 webserver.cc:460] Webserver started at http://127.17.16.124:44501/ using document root <none> and password file <none>
I20260811 09:20:17.874929 23489 fs_manager.cc:359] Metadata directory not provided
I20260811 09:20:17.875021 23489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:20:17.875285 23489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:20:17.878156 23489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/data/instance:
uuid: "093871fd479a4126a386fc73cad1958e"
format_stamp: "Formatted at 2026-08-11 09:20:17 on dist-test-slave-ncp9"
server_key: "f071ca7f5ecd781efeabc7530395a46a"
server_key_iv: "4bfe44b4602e103bdba9d7c024ebb59c"
server_key_version: "encryptionkey@0"
I20260811 09:20:17.878878 23489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/wal/instance:
uuid: "093871fd479a4126a386fc73cad1958e"
format_stamp: "Formatted at 2026-08-11 09:20:17 on dist-test-slave-ncp9"
server_key: "f071ca7f5ecd781efeabc7530395a46a"
server_key_iv: "4bfe44b4602e103bdba9d7c024ebb59c"
server_key_version: "encryptionkey@0"
I20260811 09:20:17.883832 23489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:20:17.887107 23509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:20:17.888365 23489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:20:17.888510 23489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/data,/tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/wal
uuid: "093871fd479a4126a386fc73cad1958e"
format_stamp: "Formatted at 2026-08-11 09:20:17 on dist-test-slave-ncp9"
server_key: "f071ca7f5ecd781efeabc7530395a46a"
server_key_iv: "4bfe44b4602e103bdba9d7c024ebb59c"
server_key_version: "encryptionkey@0"
I20260811 09:20:17.888679 23489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:20:17.904950 23489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:20:17.909487 23489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:20:17.909768 23489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:20:17.928624 23489 rpc_server.cc:307] RPC server started. Bound to: 127.17.16.124:40453
I20260811 09:20:17.928649 23587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.16.124:40453 every 8 connection(s)
I20260811 09:20:17.930105 23489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/data/info.pb
I20260811 09:20:17.931488 17473 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu as pid 23489
I20260811 09:20:17.931680 17473 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNGP8Lb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439649204149-17473-0/minicluster-data/master-2/wal/instance
I20260811 09:20:17.932243 17473 external_mini_cluster.cc:1428] Setting key da5be05574e75234d481ed7929bf8e40
I20260811 09:20:17.934561 23591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:20:17.942564 23591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440017, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
I20260811 09:20:17.956219 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.934108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54134 (local address 127.17.16.124:40453)
0811 09:20:17.934916 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:17.934928 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:17.934980 (+ 52us) server_negotiation.cc:407] Connection header received
0811 09:20:17.936659 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:17.936683 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:17.937145 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:17.937500 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:17.939291 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.940727 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.942466 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.943359 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.943548 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.948191 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:17.948198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:17.948208 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:17.948240 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:17.951645 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.952394 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.952403 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.952412 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.952612 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.952989 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.952995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.952999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.953663 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:17.953883 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:17.954477 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 09:20:17.954871 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":371,"threads_started":1}
I20260811 09:20:17.960860 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.945295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.126:37419 (local address 127.0.0.1:34302)
0811 09:20:17.946223 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.946276 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:17.946448 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:17.946847 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:17.946851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:17.947460 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:17.947823 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.947837 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.949084 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.949090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.950751 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.950767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.951557 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.951564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.951740 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.952307 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:17.952325 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:17.954736 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:17.958228 (+ 3492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.958236 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.958241 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.958715 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.959091 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.959095 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.959099 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.959268 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.959837 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:17.959844 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:17.960005 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 09:20:17.960151 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 09:20:17.960457 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":532,"threads_started":1}
I20260811 09:20:17.960914 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.945303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34302 (local address 127.17.16.126:37419)
0811 09:20:17.945882 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:17.945888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:17.946379 (+ 491us) server_negotiation.cc:407] Connection header received
0811 09:20:17.946575 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:17.946582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:17.946682 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:17.946816 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:17.948009 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.948851 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.950920 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.951350 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.951446 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.954933 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:17.954939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:17.954944 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:17.954983 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:17.958110 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.958825 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.958831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.958834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.958987 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.959415 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.959420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.959422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.959707 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:17.959863 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:17.960298 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 09:20:17.960512 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":287,"threads_started":1}
I20260811 09:20:17.962173 23591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.126@KRBTEST.COM
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440017, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:20:17.975454 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.959413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.17.16.126:37419)
0811 09:20:17.960089 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:17.960093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:17.960112 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:20:17.960149 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:17.960153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:17.960204 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:17.960298 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:17.961650 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.962676 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.964422 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.965132 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.965250 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.969640 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:17.969646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:17.969649 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:17.969679 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:17.972656 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.973298 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.973304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.973308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.973426 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.973790 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.973793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.973794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.974134 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:17.974332 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:17.974699 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 09:20:17.975079 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":371,"threads_started":1}
I20260811 09:20:17.975570 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.962810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.125:39551 (local address 127.0.0.1:50632)
0811 09:20:17.963045 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.963063 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:17.963153 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:17.963451 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:17.963456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:17.963918 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:17.964096 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.964108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.965408 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.965413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.966723 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.966738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.967412 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.967418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.967605 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.968057 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:17.968080 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:17.970063 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:17.973330 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.973333 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.973336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.973784 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.974157 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.974160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.974162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.974325 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.974888 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:17.974893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:17.975042 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 09:20:17.975153 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 09:20:17.975339 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260811 09:20:17.975822 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.962932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.17.16.125:39551)
0811 09:20:17.963094 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:17.963098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:17.963114 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:20:17.963264 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:17.963268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:17.963327 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:17.963402 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:17.964236 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.965269 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.966865 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.967268 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.967357 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.970246 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:17.970251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:17.970255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:17.970296 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:17.973210 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.973890 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.973895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.973898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.974061 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.974433 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.974437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.974439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.974757 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:17.974896 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:17.975271 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 09:20:17.975485 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:20:17.976688 23591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440017, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.124@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
Aug 11 09:20:17 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.16.125@KRBTEST.COM
I20260811 09:20:17.988822 23599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.977299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54140)
0811 09:20:17.977442 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:17.977459 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:17.977541 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:17.977874 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:17.977878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:17.978236 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:17.978359 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.978367 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.979300 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.979305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.980347 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:17.980362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.981268 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.981275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:17.981448 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.981977 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:17.981999 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:17.983964 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:17.986686 (+ 2722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.986691 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.986695 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.987014 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.987338 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:17.987342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:17.987346 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:17.987461 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:17.987915 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:17.987920 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:17.988064 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 09:20:17.988193 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 09:20:17.988455 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 09:20:17.989041 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.977537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54140 (local address 127.17.16.124:40453)
0811 09:20:17.977672 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:17.977676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:17.977688 (+ 12us) server_negotiation.cc:407] Connection header received
0811 09:20:17.977719 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:17.977723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:17.977768 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:17.977839 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:17.978484 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.979175 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.980467 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.981114 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.981224 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.984123 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:17.984128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:17.984131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:17.984157 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:17.986570 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.987123 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.987129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.987132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.987241 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.987560 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.987566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.987569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.987812 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:17.987931 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:17.988287 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 09:20:17.988636 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":37}
I20260811 09:20:17.990622 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:17.977167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50636 (local address 127.17.16.125:39551)
0811 09:20:17.977317 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:17.977322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:17.977412 (+ 90us) server_negotiation.cc:407] Connection header received
0811 09:20:17.977552 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:17.977557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:17.977625 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:17.977709 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:17.978808 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.979701 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.981225 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:17.981804 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:17.981920 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:17.985837 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:17.985842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:17.985845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:17.985874 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:17.988214 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.988798 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.988803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.988807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.988988 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:17.989414 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:17.989419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:17.989422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:17.989693 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:17.989896 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:17.990227 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 09:20:17.990389 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:20:17.994565 23591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e: Bootstrap starting.
I20260811 09:20:17.995265 17473 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:20:17.998456 23591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:17.999796 23591 log.cc:824] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:17.998320 23303 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5d85c85be6bb46ccbddd0f58370842cb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34310
I20260811 09:20:18.002995 23591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e: No bootstrap required, opened a new log
I20260811 09:20:18.013679 23591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } }
I20260811 09:20:18.014096 23591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093871fd479a4126a386fc73cad1958e, State: Initialized, Role: FOLLOWER
I20260811 09:20:18.014595 23591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } }
I20260811 09:20:18.015516 23603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } } }
I20260811 09:20:18.015826 23603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:18.016780 23591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:20:18.025951 23625 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:20:18.026080 23625 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:20:18.033358 23347 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } attempt: 4
I20260811 09:20:18.023517 23489 master_runner.cc:186] Error getting master registration for 127.17.16.126:37419: OK, instance_id { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" instance_seqno: 1786440016893766 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:20:18.034744 23489 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 11 09:20:18 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440016, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.126@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
I20260811 09:20:18.046965 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.033987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54150)
0811 09:20:18.034170 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:18.034190 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:18.034316 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:18.034685 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:18.034689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:18.035236 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:18.035445 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:18.035458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.036533 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.036539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:18.037520 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:18.037535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.038270 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.038275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:18.038482 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.039073 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:18.039092 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:18.041456 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:18.044550 (+ 3094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:18.044556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:18.044560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:18.044969 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:18.045309 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:18.045314 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:18.045318 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:18.045464 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:18.046003 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:18.046008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:18.046187 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 09:20:18.046298 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 09:20:18.046573 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 09:20:18.046999 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.034159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.17.16.124:40453)
0811 09:20:18.034371 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.034377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.034399 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:20:18.034476 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.034481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.034538 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.034624 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.035603 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.036387 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.037661 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.038123 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.038233 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.041645 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:18.041650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:18.041653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:18.041684 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:18.044414 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:18.045075 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:18.045081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:18.045084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:18.045199 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:18.045580 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:18.045584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:18.045586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:18.045889 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:18.046025 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.046453 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.046666 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:20:18.051604 23347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb: Bootstrap starting.
I20260811 09:20:18.055570 23347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:18.056622 23347 log.cc:824] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:18.059425 23347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb: No bootstrap required, opened a new log
I20260811 09:20:18.065405 23347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } }
I20260811 09:20:18.065804 23347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d85c85be6bb46ccbddd0f58370842cb, State: Initialized, Role: FOLLOWER
I20260811 09:20:18.066216 23347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } }
I20260811 09:20:18.067122 23629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } } }
I20260811 09:20:18.067572 23629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:18.068238 23347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:20:18.068377 23303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 09:20:18.068588 23303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:18.071990 23303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } }
I20260811 09:20:18.075294 23554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d85c85be6bb46ccbddd0f58370842cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "093871fd479a4126a386fc73cad1958e"
I20260811 09:20:18.075837 23554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:18.079311 23554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d85c85be6bb46ccbddd0f58370842cb in term 1.
I20260811 09:20:18.080602 23263 leader_election.cc:302] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093871fd479a4126a386fc73cad1958e, 5d85c85be6bb46ccbddd0f58370842cb; no voters:
W20260811 09:20:18.080922 23647 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:20:18.081137 23647 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:20:18.082566 23629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:20:18.083591 23629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [term 1 LEADER]: Becoming Leader. State: Replica: 5d85c85be6bb46ccbddd0f58370842cb, State: Running, Role: LEADER
I20260811 09:20:18.084153 23629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } }
I20260811 09:20:18.090459 23649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d85c85be6bb46ccbddd0f58370842cb. Latest consensus state: current_term: 1 leader_uuid: "5d85c85be6bb46ccbddd0f58370842cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } } }
I20260811 09:20:18.091420 23649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:18.092109 23303 leader_election.cc:288] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [CANDIDATE]: Term 1 election: Requested vote from peers ee0a4cc6a61c44dab72277817a6dc437 (127.17.16.125:39551), 093871fd479a4126a386fc73cad1958e (127.17.16.124:40453)
I20260811 09:20:18.105379 23652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:20:18.107656 23652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:20:18.086822 23229 master_runner.cc:186] Error getting master registration for 127.17.16.125:39551: OK, instance_id { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" instance_seqno: 1786440017403455 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:20:18.116819 23229 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:20:18.126978 23476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } attempt: 5
W20260811 09:20:18.135062 23272 tablet.cc:2369] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:20:18.140910 23554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [term 1 FOLLOWER]: Refusing update from remote peer 5d85c85be6bb46ccbddd0f58370842cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:20:18.142246 23649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:20:18.152405 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.125946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50646 (local address 127.17.16.125:39551)
0811 09:20:18.126146 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.126152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.127127 (+ 975us) server_negotiation.cc:407] Connection header received
0811 09:20:18.127559 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.127564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.127638 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.127707 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.130579 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.131753 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.133731 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.134438 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.134534 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.145113 (+ 10579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:18.145122 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:18.145127 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:18.145180 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:18.148965 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:18.150006 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:18.150012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:18.150016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:18.150205 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:18.150633 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:18.150638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:18.150642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:18.150998 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:18.151239 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.151800 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.152042 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 09:20:18.165781 23520 tablet.cc:2369] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 09:20:18 dist-test-slave-ncp9 krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786440017, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.16.125@KRBTEST.COM for kudu/127.17.16.124@KRBTEST.COM
I20260811 09:20:18.171068 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.114228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34314 (local address 127.17.16.126:37419)
0811 09:20:18.114429 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.114435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.125245 (+ 10810us) server_negotiation.cc:407] Connection header received
0811 09:20:18.125487 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.125493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.125605 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.125717 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.134978 (+ 9261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.136142 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.141374 (+ 5232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.142207 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.142315 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.161242 (+ 18927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:18.161251 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:18.161257 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:18.161309 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:18.165614 (+ 4305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:18.166555 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:18.166562 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:18.166566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:18.166730 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:18.167537 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:18.167543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:18.167546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:18.167909 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:18.168213 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.169719 (+ 1506us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.170341 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:20:18.177911 23603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d85c85be6bb46ccbddd0f58370842cb. Latest consensus state: current_term: 1 leader_uuid: "5d85c85be6bb46ccbddd0f58370842cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } } }
I20260811 09:20:18.178238 23603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:18.180497 23672 mvcc.cc:205] Tried to move back new op lower bound from 7317258314289922048 to 7317258314077679616. Current Snapshot: MvccSnapshot[applied={T|T < 7317258314289922048}]
I20260811 09:20:18.182261 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.127501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.16.124:40453 (local address 127.0.0.1:54172)
0811 09:20:18.127735 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 09:20:18.127752 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:20:18.127828 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:20:18.132468 (+ 4640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:20:18.132474 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:20:18.133007 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:20:18.133214 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:18.133226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.153641 (+ 20415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.153648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:18.155080 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:20:18.155097 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.166825 (+ 11728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.166833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:20:18.167097 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.167698 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:20:18.167723 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:20:18.170473 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:20:18.175786 (+ 5313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:18.175792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:18.175796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:18.176322 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:18.178581 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:20:18.178586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:20:18.178590 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:20:18.178908 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:20:18.180292 (+ 1384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:20:18.180323 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:20:18.180598 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 09:20:18.180987 (+ 389us) client_negotiation.cc:240] Negotiation successful
0811 09:20:18.181525 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 09:20:18.184507 23669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.127555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54172 (local address 127.17.16.124:40453)
0811 09:20:18.132145 (+ 4590us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.132151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.132173 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:20:18.132233 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.132239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.132308 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.132421 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.133589 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.153361 (+ 19772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.155393 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.166655 (+ 11262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.166766 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.170744 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:18.170752 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:18.170757 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:18.170798 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:18.175628 (+ 4830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:18.176457 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:18.176463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:18.176467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:18.176699 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:18.179201 (+ 2502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:18.179207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:18.179244 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:18.179989 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:18.182902 (+ 2913us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.183208 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.183680 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4440,"thread_start_us":4216,"threads_started":1}
I20260811 09:20:18.190588 23476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437: Bootstrap starting.
I20260811 09:20:18.193140 23667 mvcc.cc:205] Tried to move back new op lower bound from 7317258314289922048 to 7317258314077679616. Current Snapshot: MvccSnapshot[applied={T|T < 7317258314289922048}]
I20260811 09:20:18.199064 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.126970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54158 (local address 127.17.16.124:40453)
0811 09:20:18.127151 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.127156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.154565 (+ 27409us) server_negotiation.cc:407] Connection header received
0811 09:20:18.154788 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.154795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.154875 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.154950 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.156402 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.165475 (+ 9073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.178633 (+ 13158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.179360 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.179462 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.183162 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:20:18.183169 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:20:18.183174 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:20:18.183210 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:20:18.186806 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:18.187614 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:18.187620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:18.187623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:18.187760 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:20:18.197384 (+ 9624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:20:18.197391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:20:18.197395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:20:18.197811 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:20:18.198116 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.198402 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.198655 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65,"spinlock_wait_cycles":25472}
I20260811 09:20:18.202854 23476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437: Neither blocks nor log segments found. Creating new log.
I20260811 09:20:18.204195 23476 log.cc:824] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437: Log is configured to *not* fsync() on all Append() calls
I20260811 09:20:18.216784 23629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5d85c85be6bb46ccbddd0f58370842cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } } }
I20260811 09:20:18.217177 23629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:18.217548 23476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437: No bootstrap required, opened a new log
I20260811 09:20:18.220137 23652 catalog_manager.cc:1347] Generated new cluster ID: c9f567892e2341cd9da53148f9062a0c
I20260811 09:20:18.220250 23652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:20:18.222059 23603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5d85c85be6bb46ccbddd0f58370842cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } } }
I20260811 09:20:18.222301 23603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:18.241367 23476 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } }
I20260811 09:20:18.241874 23476 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0a4cc6a61c44dab72277817a6dc437, State: Initialized, Role: FOLLOWER
I20260811 09:20:18.242470 23476 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } }
I20260811 09:20:18.261204 23683 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } } }
I20260811 09:20:18.261730 23683 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:18.262665 23476 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:20:18.263141 23424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:20:18.267077 23424 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [term 1 FOLLOWER]: Refusing update from remote peer 5d85c85be6bb46ccbddd0f58370842cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:20:18.267758 23426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5d85c85be6bb46ccbddd0f58370842cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ee0a4cc6a61c44dab72277817a6dc437"
I20260811 09:20:18.271320 23426 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d85c85be6bb46ccbddd0f58370842cb in term 1.
I20260811 09:20:18.272521 23649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:20:18.288242 23652 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:20:18.290460 23652 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:20:18.313810 23629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5d85c85be6bb46ccbddd0f58370842cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } } }
I20260811 09:20:18.314146 23629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb [sys.catalog]: This master's current role is: LEADER
I20260811 09:20:18.313985 23697 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5d85c85be6bb46ccbddd0f58370842cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } } }
I20260811 09:20:18.314448 23697 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:18.313966 23683 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d85c85be6bb46ccbddd0f58370842cb. Latest consensus state: current_term: 1 leader_uuid: "5d85c85be6bb46ccbddd0f58370842cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d85c85be6bb46ccbddd0f58370842cb" member_type: VOTER last_known_addr { host: "127.17.16.126" port: 37419 } } peers { permanent_uuid: "ee0a4cc6a61c44dab72277817a6dc437" member_type: VOTER last_known_addr { host: "127.17.16.125" port: 39551 } } peers { permanent_uuid: "093871fd479a4126a386fc73cad1958e" member_type: VOTER last_known_addr { host: "127.17.16.124" port: 40453 } } }
I20260811 09:20:18.317879 23683 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:20:18.330829 23652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d85c85be6bb46ccbddd0f58370842cb: Generated new TSK 0
I20260811 09:20:18.335776 23652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:20:18.372757 23714 catalog_manager.cc:1259] Loaded cluster ID: c9f567892e2341cd9da53148f9062a0c
I20260811 09:20:18.378654 23714 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437: loading cluster ID for follower catalog manager: success
I20260811 09:20:18.382818 23714 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437: acquiring CA information for follower catalog manager: success
I20260811 09:20:18.392319 23714 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee0a4cc6a61c44dab72277817a6dc437: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 09:20:18.523949 23600 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.508251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.17.16.126:37419)
0811 09:20:18.510208 (+ 1957us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.510216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.521979 (+ 11763us) server_negotiation.cc:407] Connection header received
0811 09:20:18.522302 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.522327 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.522441 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.522550 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.523608 (+ 1058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34330: BlockingRecv error: recv got EOF from 127.0.0.1:34330 (error 108)
Metrics: {"server-negotiator.queue_time_us":1786}
W20260811 09:20:18.525928 23481 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.523138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50652 (local address 127.17.16.125:39551)
0811 09:20:18.523364 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.523370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.524041 (+ 671us) server_negotiation.cc:407] Connection header received
0811 09:20:18.524323 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.524329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.524424 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.524498 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.525701 (+ 1203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50652: BlockingRecv error: recv got EOF from 127.0.0.1:50652 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 09:20:18.526446 23595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.524140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54174 (local address 127.17.16.124:40453)
0811 09:20:18.524315 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.524320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.524712 (+ 392us) server_negotiation.cc:407] Connection header received
0811 09:20:18.525028 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.525035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.525131 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.525206 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.526154 (+ 948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54174: BlockingRecv error: recv got EOF from 127.0.0.1:54174 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:20:18.600170 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.578930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.17.16.125:39551)
0811 09:20:18.579152 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.579160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.592150 (+ 12990us) server_negotiation.cc:407] Connection header received
0811 09:20:18.592414 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.592421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.592676 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.592866 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:18.593757 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.594931 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.597041 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.597712 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.597816 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.598601 (+ 785us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:18.599328 (+ 727us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:18.599463 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.599717 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.599824 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 09:20:18.648218 23595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.634990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54178 (local address 127.17.16.124:40453)
0811 09:20:18.635199 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.635205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.646074 (+ 10869us) server_negotiation.cc:407] Connection header received
0811 09:20:18.646591 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.646599 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.646748 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.646822 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.647975 (+ 1153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54178: BlockingRecv error: recv got EOF from 127.0.0.1:54178 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:20:18.678996 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.665615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.17.16.125:39551)
0811 09:20:18.665850 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.665858 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.673531 (+ 7673us) server_negotiation.cc:407] Connection header received
0811 09:20:18.673787 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.673797 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.673898 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.674005 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:18.674482 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.675356 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.676853 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.677449 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.677577 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.678076 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:18.678234 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:18.678320 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.678540 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.678635 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 09:20:18.687809 23595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.686226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42848 (local address 127.17.16.124:40453)
0811 09:20:18.686409 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.686415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.686437 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:20:18.686516 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.686523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.686619 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.686718 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.687558 (+ 840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42848: BlockingRecv error: recv got EOF from 127.0.0.1:42848 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:20:18.706354 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.699005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.17.16.125:39551)
0811 09:20:18.699227 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.699235 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.701437 (+ 2202us) server_negotiation.cc:407] Connection header received
0811 09:20:18.701659 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.701664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.701735 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.701814 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:18.702324 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.703155 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.704515 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.705007 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.705126 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.705538 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:18.705654 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:18.705730 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.705879 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.705971 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 09:20:18.714665 23595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.712612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42852 (local address 127.17.16.124:40453)
0811 09:20:18.712810 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.712815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.713184 (+ 369us) server_negotiation.cc:407] Connection header received
0811 09:20:18.713364 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.713368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.713443 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.713535 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.714441 (+ 906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42852: BlockingRecv error: recv got EOF from 127.0.0.1:42852 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:20:18.728305 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.722223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35052 (local address 127.17.16.125:39551)
0811 09:20:18.722501 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.722509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.722979 (+ 470us) server_negotiation.cc:407] Connection header received
0811 09:20:18.723174 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.723181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.723276 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.723399 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:18.723791 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.724977 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.726330 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.727034 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.727165 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.727497 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:18.727640 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:18.727732 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.727818 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.727897 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
W20260811 09:20:18.737324 23595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.735265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42866 (local address 127.17.16.124:40453)
0811 09:20:18.735578 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.735584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.735941 (+ 357us) server_negotiation.cc:407] Connection header received
0811 09:20:18.736100 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.736107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.736225 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.736345 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.737064 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42866: BlockingRecv error: recv got EOF from 127.0.0.1:42866 (error 108)
Metrics: {"server-negotiator.queue_time_us":166}
I20260811 09:20:18.754402 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.748649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35068 (local address 127.17.16.125:39551)
0811 09:20:18.748861 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.748869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.749252 (+ 383us) server_negotiation.cc:407] Connection header received
0811 09:20:18.749407 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.749416 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.749511 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.749623 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:18.749966 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.751109 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.752394 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.753106 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.753247 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.753605 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:18.753782 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:18.753883 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.753959 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.754063 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 09:20:18.761838 23595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.759922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.17.16.124:40453)
0811 09:20:18.760158 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.760164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.760453 (+ 289us) server_negotiation.cc:407] Connection header received
0811 09:20:18.760604 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.760610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.760708 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.760815 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.761609 (+ 794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42878: BlockingRecv error: recv got EOF from 127.0.0.1:42878 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:20:18.786989 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.781028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35084 (local address 127.17.16.125:39551)
0811 09:20:18.781241 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.781248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.781620 (+ 372us) server_negotiation.cc:407] Connection header received
0811 09:20:18.781800 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.781807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.781907 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.782060 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:18.782465 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.783674 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.785045 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.785708 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.785844 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.786253 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:18.786396 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:18.786507 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.786600 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.786695 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 09:20:18.795589 23595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.793566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42886 (local address 127.17.16.124:40453)
0811 09:20:18.793743 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.793749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.794186 (+ 437us) server_negotiation.cc:407] Connection header received
0811 09:20:18.794377 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.794382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.794472 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.794558 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.795355 (+ 797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42886: BlockingRecv error: recv got EOF from 127.0.0.1:42886 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 09:20:18.836270 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.830911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35088 (local address 127.17.16.125:39551)
0811 09:20:18.831108 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.831114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.831663 (+ 549us) server_negotiation.cc:407] Connection header received
0811 09:20:18.831886 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.831891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.831964 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.832044 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:18.832483 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.833260 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.834568 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.835015 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.835128 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.835547 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:18.835661 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:18.835735 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.835885 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.835973 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 09:20:18.844193 23595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.842519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.17.16.124:40453)
0811 09:20:18.842705 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.842711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.842937 (+ 226us) server_negotiation.cc:407] Connection header received
0811 09:20:18.843082 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.843089 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.843182 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.843276 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.843944 (+ 668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42896: BlockingRecv error: recv got EOF from 127.0.0.1:42896 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:20:18.916886 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:18.911670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35098 (local address 127.17.16.125:39551)
0811 09:20:18.911851 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.911858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.912243 (+ 385us) server_negotiation.cc:407] Connection header received
0811 09:20:18.912433 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.912438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.912509 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.912587 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:18.913090 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.913835 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.915207 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:18.915630 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:18.915749 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:18.916168 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:18.916294 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:18.916370 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:18.916517 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 09:20:18.916598 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 09:20:18.924293 23595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:20:18.922422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42906 (local address 127.17.16.124:40453)
0811 09:20:18.922658 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:18.922665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:18.923097 (+ 432us) server_negotiation.cc:407] Connection header received
0811 09:20:18.923271 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:18.923276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:18.923345 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:18.923421 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:20:18.924086 (+ 665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42906: BlockingRecv error: recv got EOF from 127.0.0.1:42906 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:20:19.028107 23625 catalog_manager.cc:1259] Loaded cluster ID: c9f567892e2341cd9da53148f9062a0c
I20260811 09:20:19.028218 23625 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e: loading cluster ID for follower catalog manager: success
I20260811 09:20:19.030700 23625 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e: acquiring CA information for follower catalog manager: success
I20260811 09:20:19.032418 23625 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 093871fd479a4126a386fc73cad1958e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:20:19.061183 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:19.056275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35114 (local address 127.17.16.125:39551)
0811 09:20:19.056484 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:19.056491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:19.056784 (+ 293us) server_negotiation.cc:407] Connection header received
0811 09:20:19.057011 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:19.057016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:19.057090 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:19.057172 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:19.057619 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:19.058312 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:19.059603 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:19.059993 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:19.060101 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:19.060475 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:19.060594 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:19.060667 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:19.060813 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 09:20:19.060920 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:20:19.071899 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:19.066494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.17.16.124:40453)
0811 09:20:19.066690 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:19.066696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:19.066934 (+ 238us) server_negotiation.cc:407] Connection header received
0811 09:20:19.067126 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:19.067131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:19.067280 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:19.067380 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:19.067764 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:19.068431 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:19.069793 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:19.070209 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:19.070323 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:19.070804 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:19.071334 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:19.071467 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:19.071574 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 09:20:19.071657 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:20:19.084218 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:19.077092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.17.16.126:37419)
0811 09:20:19.077502 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:19.077508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:19.077627 (+ 119us) server_negotiation.cc:407] Connection header received
0811 09:20:19.078006 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:19.078013 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:19.078272 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:19.078439 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:19.079145 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:19.080295 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:19.082204 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:19.082600 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:19.082718 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:19.083176 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:19.083607 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:19.083714 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:19.083826 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 09:20:19.083914 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":219,"threads_started":1}
I20260811 09:20:19.084358 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:19.077799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35116 (local address 127.17.16.125:39551)
0811 09:20:19.077983 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:19.077988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:19.078199 (+ 211us) server_negotiation.cc:407] Connection header received
0811 09:20:19.078454 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:19.078459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:19.078538 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:19.078598 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:19.079408 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:19.080451 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:19.082520 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:19.083147 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:19.083235 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:19.083696 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:19.083822 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:19.083897 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:19.084088 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 09:20:19.084166 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:20:19.085572 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:20:19.078298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42922 (local address 127.17.16.124:40453)
0811 09:20:19.078481 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:20:19.078487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:20:19.079268 (+ 781us) server_negotiation.cc:407] Connection header received
0811 09:20:19.079491 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:20:19.079497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:20:19.079567 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:20:19.079649 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:20:19.080190 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:19.081175 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:19.083247 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:20:19.083825 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:20:19.083934 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:20:19.084504 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:20:19.084625 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:20:19.084688 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 09:20:19.085224 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 09:20:19.085308 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:20:19.088593 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 23229
I20260811 09:20:19.089615 23339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:20:19.200990 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 23229
I20260811 09:20:19.226359 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 23356
I20260811 09:20:19.227439 23469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:20:19.333726 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 23356
I20260811 09:20:19.360656 17473 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu pid 23489
I20260811 09:20:19.361910 23582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:20:19.469781 17473 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNGP8Lb/build/asan/bin/kudu with pid 23489
2026-08-11T09:20:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4600 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9179 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (370194 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST